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Message from 
the Director 



I am pleased to present the annual 
report on provincial achievement 
testing for the June 1993 
administration. Students were 
assessed in Grade 3 language 
learning, Grade 6 social studies, 
and Grade 9 science. 

As in past years, the information in 
this report is based only on the 
students in the province who wer° 
assessed. However, approximately 
10 percent of the students in each 
grade did not write the test 
(page 1), so we have no information 
about these students' achievement 
in relation to provincial standards. 
Beginning in the 1994-95 school 
year, all students registered in 
grades 3, 6, and 9 programs are 
expected to write the tests or be 
reported as not being able to 
achieve standards. Following the 
1995 adminstraiion, we will be able 
to provide more comprehensive 
information to schools andsfo report 
to the public on the achievement of 
all students, but our new inclusion 
policy is likely to affect the 
percentage of students who meet 
provincial standards. 

School level results suggest 
declining performance across 
grades 3, 6, and 9 as evident in the 
percentage of schools meeting the 
provincial achievement standard. 
As shown on page 56, Grade 9 level 
data show that a satisfactory 
percentage of students met the 
achievement standard in only two 
out of three schools. This indicates 
that not enough Grade 9 students 
know their science in one- third of 
our schools. The information in this 



report, together with detailed school 
data provided separately, should 
help school administrators, 
teachers, and the community to 
identify what needs to be done to 
improve learning in their schools. 

Again this year, the assessments 
included performance tasks at each 
grade level. These tasks extend the 
range of areas assessed and 
generally raise additional questions 
of importance for classroom 
instruction. For example, the Grade 
3 results, presented in Section 2, 
show that fewer students clearly 
xnet the provincial standard for 
writing in the performance 
assessment than on the written 
component of the achievement test. 
This type of extended writing task 
may be more useful in the context of 
classroom assessment, as a means 
of gaining insights about a 
student's learrung processes in 
writing for diagnostic purposes, 
since it does not add much to tlie 
measure of student's writing 
competence at the end of Grade 3. 

As part of the 1993 testing, a 
special study was done with 
students aged 9 to investigate the 
merit of age-based testing. The 
study included a survey of teachers 
on the desirability of assessment by 
age. Although the results show 
some useful information about age 
and grade level performance, 
assessment by age proved 
disruptive for some schools. The 
study supports our decision to 
continue with our grade-based 
approach to assessment. See 
Section 9 for more detail. 



The provincial assessments and 
special studies can only be carried 
out with the help of teachers, 
principals, and superintendents. 
On behalf of the staff at the 
Student Evaluation Branch, I want 
to express our appreciation for this 
help. I trust that the enclosed report 
will be interesting to all of you, and 
I hope it will assist you in reflecting 
on your instructional programs and 
encouraging all students to meet 
high standards of achievement. A 
questionnaiw is included at the end 
of the report, and we welcome your 
feedback. 




Frank G. Horvath, Director 
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Sec-lion 1 



Summary of 
Achievement Test Results 



Parents, educators and the general 
public need to know how well 
Alberta students are achieving in 
relation to provincial standards. 

The 1993 achievement test results* 
show what students in grades 3,6, 
and 9 .aiow and can do in Grade 
Level 0 Language Learning, Grade 
6 Social Studies, and Grade 9 
Science. 

Knowledge of what is possible 
produces new enthusiasm, 
raises sights, establishes new 
challenges and ultimately can 
improve personal and societal 
performance.** 

This first section of the report 
describes certain broad 
characteristics of the student 
population who wrote the 
achievement tests and provides 
answers to the following questions: 

• How many students wrote 
the achievement test for 
their grade and how does 
this compare to 1992? 



• What percentage of the 
students achieved the 
acceptable standard 
according to criteria set 
by Alberta Education? 

• What percentage of the 
students achieved the 
standard of excellence 
according to criteria set 
by Alberta Education? 

Student Participation 

How many students wrote the 
achievement test for their 
grade and how does this 
compare to 1992? 

In June 1993, principals reported 
a total population of 116 296 
students in the grades tested. 
There were 42 837 students in 
Grade 3, 39 097 students in 
Grade 6, and 34 362 students in 
Grade 9. Of the to*;al population, 
102 702 wrote achievement tests. 
This means that 88.3% of students 
reported by principals to be 



enrolled in schools in June wrote 
the tests. The remaining 13 594 
students were exempted from 
writing or were absent the day the 
tests were administered. 

Figure 1-1 shows student 
participation data for June 1993 
and for June 1992. This report does 
not take these absent or exempted 
students into account in answering 
the questions about the percentage 
of students who achieved the 
acceptable standard and the 
standard of excellence. 



In some cases, the percentages in a chart may not add to 100 because of rounding. 

Learning Mathematics/Learning Science, International Assessment of Educational Progress, Educational Testing Service, 
February 1992. 
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Figure 1-1 

Student Participation in Achievement Tests* 

June 1992 and June 1993 

Grade 3 
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Grade 6 



Grade 9 






1992 
/V=-.41 696 
Social 
Studies 



1993 
N = 42 837 
English 
Language 
Learning 



1992 
N=37 837 
English 
Language 
Arts 



1993 
N=39 09', 
Social 
Studies 



1992 
A/ =34 386 
Mathematics 



1993 
/V=34 362 
Science 



mm 



Students writing, June 1992 



Students absent or exempt, June 1 992 



Students writing, June 1993 



Students absent or exempt, June 1 993 



*The number of students writing the tests includes students in the regular English program, and Francophone and French Immersion students 
who wrote the achievement test either in English or in French translation. 
N= number of students reported by principals to be enrolled in June. 



Observations and 
Discussion 

In 1993, the percentage of students 
who wrote the test was close to 
90% for each of the three grades. 
The percentage for Grade 6 was 
the highest (89.7%) compared to 
87.5% for Grade 3 and 87.7% for 
Grade 9. 

In terms of absolute numbers, 
Grade 3 had the highest number of 
students (37 500) writing the test, 
followed by students in Grade 6 
(35 062) and students in Grade 9 
(30 140). This trend corresponds 
with the trend in student 
enrollment by grade reported as of 
September 30. 

The total number of grades 3, 6, 
and 9 students who wrote the tests 
increased from 97 315 in 1992 to 
102 702 in 1993. This represents an 
increase of 5.5%. The increase 



occurred at each grade level in both 
absolute numbers and percentages. 
The percentage of students who 
were absent or exempted from 
writing the tests decreased in 1993 
compared to the percentage for 
1992. The 1993 overall 
participation rate of 88.3% is an 
improvement over the 1992 
participation rate of 85.4%. 

Results in Relation 
to Standards for 
Students Who Wrote the 
Achievement Tests 

What percentage of the 
students achieved the 
acceptable standard according 
to criteria set by Alberta 
Education? 

What percentage of the 
students achieved the standard 
of excellence according to 



criteria set by Alberta 
Education? 

Results are reported in terms 
of three related but different 
standards: curriculum standards, 
assessment standards, and 
achievement standards. 

Curriculum standards are the 

broad learnings, sequenced into 
grades, that students are expected 
to have achieved. They include 
statements of knowledge, skills, 
and attitudes against which 
student achievement is to be 
judged. 

Assessment standards are the 

scores to be achieved by a student 
on a specific test or part of a test 
before the performance of that 
student is judged to be 
"acceptable" or "excellent" relative 
tc the curriculum standards. 
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Alberta Education reports the 
results for the achievement tests in 
relation to an acceptable standard 
and a standard of excellence. 

Achievement standards state 
what percentage of students at a 
given grade in school is expected to 
achieve the acceptable standard or 
the standard of excellence. These 
achievement standards apply to 
school, jurisdiction, and provincial 
performance. 

Assessment and achievement 
standards for the 1993 achievement 
tests were based on 
recommendations from a 
curriculum and test development 
committee, a test review 
committee, a public advisory 
committee, and experienced 
teachers who applied standard- 
setting procedures to the tasks 
under the guidance of the Analytic 
Services Unit of the Student 
Evaluation Branch. An outline of 
the processes followed is provided 
in Appendix A. Appendix D 
describes the public review of 
standards and results. 

Guidelines for interpreting the 
1993 results are given in 
Appendix C, and Appendix D 
provides answers to a number of 
questions frequently posed by 
parents. Appendix E describes the 
development of achievement tests. 

In this section, results are reported 
for the total test. Results for the 
major components of the tests can 
be found in sections 2, 3, and 4. 

From discussions with educators, 
test development specialists and 
curriculum specialists, and based 
on our experience with measuring 
student achievement according to 
the expectations in the Progrt m of 
Studies, we expect 85% of students 
who write the test to achieve the 
acceptable standard and 15% of 



Figure 1-2 

Percentage of Students Achieving Acceptable Standard* on the 
Total Test 

June 1993 



100 




Language Learning Social Studies Science 

(Gr. Level 3) (Gr. 6) (Gr. 9) 

N= 37 500 A/= 32 630 N= 28 613 



includes students achieving the standard of excellence 

"85% of students who wrote the test were expected to achieve the acceptable standard on 
the total test. 

Figure 1-3 

Percentage of Students Achieving Standard of Excellence on the 
Total Test 

June 19b3 



20 




Language Learning Social Studies Science 

(Gr. Level 3) (Gr. 6) (Gr. 9) 

N = 37 500 N= 32 630 N= 28 613 



*15% of students who wrote the test were expected to achieve the standard of excellence on 
the total test. 

v : >::>:;:^ 



students who write the test to 
achieve the standard of excellence. 

Figures 1-2 and 1-3 present 
the percentage of students who 
achieved the acceptable standard 
and the standard of excellence 
based on their total test scores. 

Provincial results for Grade 3 in 
1993 are based on test scores 
achieved by all students who wrote 



the test, including Francophone 
and French Immersion students. 

Provincial results for grades 6 and 
9 in 1993 are based on test scores 
achieved by students who were in 
the regular English program. 
Results for Francophone and 
French Immersion students were 
reported directly to participating 
schools and jurisdictions and are 
not calculated in provincial results 
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General Observations 
on Results 



Of the three grades and subjects 
tested in 1993, the results in 
Grade Level 3 Language Learning 
were closest both to reaching 
provincial achievement standards 
and to being judged satisfactory. In 
Grade Level 3 Language Learning, 
results based on the total test 
scores revealed that the percentage 
of students achieving the 
acceptable standard was slightly 
lower than expected, and the 
percentage achieving the standard 
of excellence was marginally lower 
than expected. 

Results for Grade 6 Social Studies 
were disappointing. The 
percentages of students achieving 
standards were lower than 
expected for both levels. 

Grade 9 Science results were 
mixed. The percentage of students 
achieving the acceptable standard 
was below expectation. The 
percentage of students who 
achieved the standard of excellence 
was higher than expected. 



Section 2 

* Grade Level 3 

Language learning 

V T 



In 1993 a number of assessments 
were carried out in Grade Level 3 
Language Learning. The 
achievement test was administered 
to students province-wide. A 
Performance-Based Assessment 
was administered to a sample of 
Grade Level 3 Language Learning 
students. As well, a sample of 
students and teachers participated 
in a study to examine the 
relationship among various 
contexts for learning and 
achievement. The results from all 
these assessments follow. 



Achievement Test 
General Description 

The Grade Level 3 Language 
Learning Achievement Test had 
two parts. Part A: Writing was a 
65-niinute assignment consisting of 
a picture and a prompt for writing. 
The assessment was designed to 
encourage students to use the 
writing process. Part B: Reading 
was a 50-minute reading test 
consisting of 40 multiple-choice 
questions based on nine reading 
selections. 

Statistics for the total test and for 
the components are based on the 
results achieved by 37 500 
students: 34 607 wrote the regular 
form and 2 893 participated in the 
achievement-over-time study, as 
shown in Table 2-3. This section of 
the report answers the following 
questions: 



• How many Grade Level 3 
students wrote each form of the 
test or were absent or exempt? 

• What percentage of students who 
wrote the Grade Level 3 
Language Learning achievement 
test achieved the acceptable 
standard? 

• What percentage of students who 
wrote the Grade Level 3 
Language Learning achievement 
test achieved the standard of 
excellence? 

• What did Grade Level 3 students 
know and what could they do in 
Language Learning? 

• What parts of the Language 
Learning curriculum were 
difficult for Grade Level 3 
students? 

Summary of Results 

Results show that 82.2% of 
students who wrote the test 
achieved the acceptable standard 
and 14.1% achieved the standard 
of excellence on the total test. 
These results were slightly lower 
than expected. 

Content of the Test 

Part A: Writing provided the 
students with a picture and a brief 
explanation as a prompt for their 
writing. Students chose the format 
(narrative, letter, or diary/journal 
entries) that would allow them to 



do their best writing while using 
their imagination and background 
experience. This part of the te3t 
was scored in three categories: 
Content and Development, Use of 
Language, and Conventions. 
Content and Development was 
weighted to be worth twice as 
much as each of the other 
categories. 

Part B: Reading questions were 
based on reading selections from 
fiction, non-fiction, poetry, and 
visual media. These selections 
were chosen to reflect the varying 
interests and ability levels of 
students in Grade Level 3 
classrooms. Canadian material was 
used wherever possible. 
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Teat Blueprint 

The test blueprint for Part A: 
Writing shows the reporting 
categories (scoring guide), a 
description of the writing 
assignment, and the emphasis of 
communication that was assessed. 



7 able 2-1 

Grade Level 3 Language Learning 
Achievement Test Blueprint 
Part A: Writing 

June 1993 



Reporting Category (Scoring Guide) 1 Description of Writing Assignment Standards 



Content and Development* 


I 


1 —meets or exceeds 


Events and/or actions should be plausible and appropriate 


j The writing assignment follows a writing 


> the standard of 


to the student's purpose for communicating. The student 


j prompt that is read aloud to the students. 


j excellence 


should be able to describe characters and settings that are 


| The assignment allows the student to select 


I —approaches the 


appropriate within the context or terms of reference 


j the format that would best fit his/her 


> standard of 


established by the student. 


i approach to the prompt 


I excellence 




i 


! —clearly meets the 


Use of Language 


! 


! acceptable 


The student should be able to use words and expressions 


i 


| standard 


effectively in writing. 


i 


' —approaches the 






j acceptable 


Conventions (Using the conventions of language correctly 




i standard 


and effectively) 




—clearly below the 


The student should be able to communicate clearly in writing 




! acceptable 


by adhering to appropriate spelling, grammar, punctuation, 




| standard 


and capitalization. 




| INS - Insufficient 



* This category is weighted to be worth twice as much as each of the other two. 
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The test blueprint for Part B.- 
Reading shows the reporting 
categories and genre under which 
questions are classified. 



Table 2 2 

Grade Level 3 Language Learning 
Achievement Test Blueprint 
Part B: Reading 

June 1993 



Organizational 
Framework 
(Functions) 



General Concept Statement 



Question Distribution by Genre 
Specific Fiction 
Learner Visual Non* 

Expectations Stories Poetry Media Fiction 



Approx. Approx. 
Number Percent 
of Items of Test 



Exploring Concept D: Knowing how ideas 
and information can be 
organized and presented 
contributes to the enhanced 
understanding and 
communication of ideas. 



D.1, D.2, D.3, 
D.4, D.5, D.6, 
D.7 



28 



32 



14, 16 



8, 37, 38, 
39, 40 



22.5 



Constructing Concept G: The ability to make 
associations and connections is 
essential to the understanding 
and communication of meaning. 



G.1, G.2, G.3, 1. 19, 20, 
G.5, G.6 27, 31 



33,34 



17.5 



Concept H: The ability to think 
analytically is necessary for 
critical reading, listening, and 
writing.. 



H.1.H.3, H.6, 2,4,5,18, 35,36 

H.9, H.10, 30 

H.11.H.12 



13, 15 12,22,24 



12 



30 



Concept I: The ability to 
synthesize is necessary to the 
understanding and retention of 
ideas and information. 



1.1, 1.3, 1.5 



3, 21, 29 



17 



6, 7,9, 10, 
11,23, 25, 
26 



12 



30 



Total Items 
Total Percent 



14 

35 



5 

12.5 



5 

12.5 



16 
40 



40 
100 



100 
100 
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Student Participation 

In June 1993, principals reported a 
total population of 42 837 students 
in Grade Level 3. Table 2-3 
presents the number and 



percentage distribution of students 
who wrote the Grade Level 3 
Language Learning Achievement 
Test, and who were absent or were 
exempted. In total, 87.5% of the 



students reported to be in Grade 
Level 3 in June 1993 wrote the 
Grade Level 3 Language Learning 
Achievement Test. 



Table 2-3 

Grade Level 3 Language Learning 
Student Participation 

June 1993 



j 

Category ; 


Number 
of Students 


Percentage 
of Students 


Total Number of Students in Regular Programs* 




37 500 


87.5 


Students Who Wrote the Regular Form** 


34 607 




80.8 


Students Who Wrote the Achievement-Over-Time Forms: 
Form M (Lavender)— Same as the Regular Form 
Form L (Red) 
Form K (Blue) 


2 893 
1 013 
940 
940 




6.8 
2.4 
2.2 
2.2 


Students Absent 




2 299 


5.4 


Students Exempted from Writing 




3 038 


7.1 


I 

Categories of Exemption: 








1. Special Needs Students 

2. Course Was Not Taught This Term 

3. English as a Second Language Students 

4. Other (as approved by the Superintendent) 


1 608 
61 
604 
765 




3.8 
0.1 

! 1.4 
1.8 


Total Principals' Reported Population 
1 Test Day, J u ne 1 993 : 


I 


42 837 


100.0 


! Grade 3 Enrollment: September 30, 1992 
i 


42 637 





* Provincial results are based on test scores achieved by all students who wrote the Grade Level 3 Language Learning test, including 
Francophone students, French Immersion students, and students in other programs. 

** Regular Form refers to the unmodified 1993 test. The modified tests are described in Section 8 of this report. 
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Results in Relation 
to Standards 

Achievement test results are 
reported in relation to assessment 
and achievement standards. The 
Provincial Assessment 
Standard is the lowest score on a 
test that a student must achieve 
for his/her performance to be 
judged "acceptable* and/or 
"excellent" in relation to curricular 
expectations. The Provincial 
Achievement Standard refers to 
the percentage of students writing 
the test who are expected to 
achieve the Provincial Assessment 
Standard. 

Figures 2-1 and 2-2 and Table 2-4 
show the percentage of 3tudents 
achieving the acceptable standard 
and the standard oy excellence on 
the total test and on components of 
the test. The levels of performance 
in Grade Level 3 writing and 
reading were marginally lower 
than expected; however, 
significantly more students than 
expected achieved the standard of 
excellence for reading. 

When interpreting these findings, 
it is important to remember that 
they are based on the test scores of 
only a portion (87.5%) of the 
students reported to have been 
enrolled in Grade Level 3 in June 
1993, i.e., those students in regular 
programs who wrote the 
achievement test. These figures 
tell us nothing about the levels of 
achievement of students who were 
absent on the day of writing, or of 
those students who were exempted 
from writing. 



Figure 2-1 

Grade Level 3 Language Learning 

Percentage of Students Achieving Acceptable Standard on the 
Total Test and on Components of the Test 

June 1993 



100 




Writing Skills Reading Skills Total Test 



•Provincial Achievement Standard— 85% of the students in the province who wrote the test 
are expected to achieve the acceptable standard 



Figure 2-2 

Grade Level 3 Language Learning 

Percentage of Students Achieving Standard of Excellence on the 
Total Test and on Components of the Test 

June 1993 




•Provincial Achievement Standard— 15% of the -tudents in the province who wrote the test 
are expected to achieve the standard of excellence 



Table 2-4 

Grade Level 3 Language Learning 

Students Achieving Standards on the Total Test and on Components of the Test 

June 1993 







Provincial 


Provincial 


Students Achieving Assessment 




Maximum 


Assessment 


Achievement 




Standard 






Possible 


Standard* 


Standard** 


Expected 


Actual 


Actual 


Reporting Category 


Score 


(Raw Score) 


(Percent) 


Number 


Number 


Percent 


Standard of Excellence 














1 Total Test 


100 


79 


15 


5 625 


5 271 


14.1 


Writing Skills Only 


20 


15 


15 


5 625 


5 432 


14.5 


Reading Skills Only 


40 


33 


15 


5 625 


10 788 


28.8 


Acceptable Standard 














Total Test 


100 


48 


85 


31 875 


30 827 


82.2 


Writing Skills Only 


20 


9 


85 


31 875 


30 499 


81.3 


Reading Skills Only 


40 


19 


85 


31 875 


30 809 


82.2 


Below Acceptable Standard 


N/A 


N/A 


N/A 


N/A 


3 251 


8.7 


on Bo; . Writing and Reading Skills 















The Provincial Assessment Standard is a score determined by standard-setting procedures and is the lowest score a student must 
achieve for his/her performance to be judged "acceptable" and/or "excellent" in relation to curricular expectations. See Appendix A. 

"The Provincial Achievement Standard refers to the percentage of students expected to achieve the Provincial Assessment Standard. 



It should be noted that the actual 
percentages of students meeting 
standards on the total test (14.1 
and 82,2) are based on the 
37 500 students in the regular 
programs who wrote the test. 

If, however, the percenta; ?s are 
based on the total population 
reported by principals (42 837), the 
percentages achieving standards 
on the total test would be: 

12,3% meeting the standard of 

excellence, 

72,0% meeting the acceptable 
. standard. 

If the percentages are based on the 
September 30, 1992, Grade Level 3 
enrollment (42 637), the 
percentages achieve the standards 
on the total test would be: 

12.4% achieving the standard of 

excellence, 

72.3% achieving the acceptable 
standard. 



It is emphasized that the above 
percentages, based on total 
population and enrollment figures, 
present the lowest estimate of 
achievement. It is highly likely 
that some of the students who were 
absent, exempt, or not accounted 
for would have achieved standards. 
The absence of information on 
these students is nonetheless 
problematic. 

The number of students achieving 
the acceptable standard and the 
standard of excellence for each 
school was analyzed to determine 
whether schools were below, 
meeting, or exceeding provincial 
achievement standards. Schools 
classified as meeting provincial 
achievement standards were those 
for which the difference between 
the actual number of students and 
the expected number of students at 
or above standards was not 
statistically significant. Differences 



are only reported when there is a 
5% or smaller probability that a 
difference of that aize could occur 
by chance. 

About one-quarter of schools were 
significantly below the provincial 
achievement standard. School 
administrators and teach 3rs 
should critically examine language 
learning achievement test results 
to determine what they are going 
to do to improve the level of 
student achievement. Although 
this is true for all schools, it is 
particularly important for those 
"hools performing below the 
provincial achievement standard. 
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Table 2 5 

Grade Level 3 Language Learning 

Percentage Distribution of Schools* Meeting Provincial Achievement Standards on the Total Test 
(N=1106) 

June 1993 





Percentage Distribution of Schools 


Standard 


Significantly Below 
Provincial Achievement 
Standard 


Not Significantly Different 
From Provincial 
Achievement Standard 


Significantly Above 
Provincial Achievement 
Standard 


Standard of Excellence 


7.1 


83.1 


9.8 


Acceptable Standard 

i 


23.1 


69.0 


7.9 



'Schools with fewer than five students are excluded because the statistical significance of the difference between the number achieving the 
standard and the number expected to achieve the standard when calculated and reported is not educationally meaningful. 



Results for 
Part At Writing 

Results for Part A: Writing are 
most clearly understood in the 
context of the assignment students 
responded to and in the context of 
the scoring guides. The average 
raw score was 11.4 out of a possible 
20, with a standard deviation of 
3.1. Coii. pie te scoring guides are 



available from the Student 
Evaluation Branch, Alberta 
Education. 

Although the papers were scored 
on a one-marker system, 200 
randomly selected papers were re- 
marked so that a second set of 
scores was available to confirm 
scoring consistency. Of the scores 



awarded on the second reading, 
49.3% were identical to the original 
score on the same scale and 42.3% 
varied by only one point. The one- 
marker system produces results 
that are reasonably reliable for 
groups of 25 or more students. 
Achievement test scores, however, 
are less reliable for individual 
students. 
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The results presented in Table 2-6 
are best considered in terms of the 
percentage of students that 
markers judged to have presented 
work that clearly met the 
acceptable standard for any 
reporting category. It is possible to 
draw conclusions about local 
program strengths and weaknesses 
by comparing local results to 
provincial standards. Statistics 
presented in Table 2-6 are based 
on results achieved by 37 500 
students. 

It should be noted that writing 
standards in 1993 were higher 
than those of previous years. In 
1993, for example, to be awarded a 



3 a student's performance was 
described as "clearly" meeting 
expectations, whereas in previous 
years a score of 3 reflected 
"satisfacto y" performance. 

Nonetheless, the results in the 
three individual reporting 
categories bear close scrutiny. Of 
these, in "Content and 
Development" (students describe 
characters and settings that are 
appropriate for the context or 
terms of reference they have 
chosen), approximately 1 in 3 or 
32.3% were not able to meet the 
acceptable standard. In "Use of 
Language" (students use words 
and expressions effectively in their 



writing), 21 7% or slightly more 
than 1 in 4 did not meet the 
acceptable standard. The results for 
the "Conventions" category 
(students communicate clearly by 
adhering to appropriate spelling, 
grammar, punctuation, and 
capitalization) are similar to the 
"Content and Development" 
category. A large number of 
students, slightly more than 1 in 3 
or 35.8%, did not meet the 
acceptable standard. Students did 
better on the writing component of 
the achievement test than they did 
on the performance-based 
assessment (see Table 2-10). 



Table 2-6 

Grade Level 3 Language Learning 

Percentage Distribution of Scores by Reporting Category 
Part A: Writing 

June 1993 





- — ■ — i 

Reporting Category 


Standards 


Content and 
Development 


Use of Language 


| 

Conventions i 


Meets or Exceeds the Standard of Excellence 


Z9 


3.3 


4.4 


] Approaches the Standard of Excellence 14.7 


15.4 


17.7 

, 


] Ciearly Meets the Acceptable Standard ! 50.1 


52.6 


42.0 


i Approaches the Acceptable Standard 


27.3 


24.7 


29.1 


j Clearly Below the Acceptable Standard 




3.8 


6.5 


i INS (Insufficient) 

t 


02 




02 


02 



Results for 
Part B: Reading 
Reporting Categories 

Table 2-7 shows the total marks 
possible and the provincial raw 
score results for the reporting 
categories of the reading portion of 
the Grade Level 3 Language 
Learning Achievement Test. 



It is important to stress that the 
averages on the various reporting 
categories cannot be directly 
compared with one another. 
Rather, the results shown in Table 
2-7 can best be used in conjunction 
with parallel tables in the 
jurisdiction and school reports. 
Variatic ns in patterns of students* 
responses to questions can help to 



indicate strengths and weaknesses 
in local educational programs. 
Statistics presented in this table 
are based on results achieved by 
35 620 students (those who wrote 
the regular form an^ those who 
wrote Form ft 1 



Table 2 7 

Grade Level 3 Language Learning 

Raw Score Results by Reporting Category 

Part B: Reading 

June 1993 





Number of 


Raw Score 


Raw Score 


Reporting Category 


Questions 


Average 


Standard Deviation 


Total 


40 


26.7 


8.0 


Organization of Information 


9 


6.0 


2.2 


Associating/Connecting 


7 


4.3 


1.8 


Analyzing 


12 


8.4 


2.5 


| Synthesizing 


12 


8.0 


2.7 


j Stories 


14 


9.3 


3.2 


| Poetry 


! 5 


3.1 


1.4 


j Visual Media 




3.8 


1.2 


I Non-Fiction 


16 


10.5 

I 


3.5 



Percentage of Students 
Choosing Each Alternative 

Table 2-8 shows the percentage of 
students who chose each 
alternative (A, B, C, and D) for 
each multiple-choice question. The 
correct response for each question 
is identified with an asterisk and 
the curriculum standard each 



question measures is specified. 
The questions are grouped by 
reporting category. 

The results shown in Table 2-8 can 
best be used in conjunction with 
similar tables in the jurisdiction 
and school reports. Variations in 
patterns of students' responses to 



questions can help to indicate 
strengths and weaknesses in local 
education programs. 

Statistics presented in Table 2-8 
are based on results achieved by 
35 620 students (those who wrote 
the regular form and those who 
wrote Form M). 



Table 2 8 

Grade Level 3 Language Learning 

Results for Individual Multiple-Choice Questions** 

June 1993 



item 



Distribution of Responses (%) 



Category 



Curriculum Standard 





A I 


B 


! c ! 


D ! 




i 
I 


8 


57.2* ; 


4.1 


; I 

i 68 I 


30.8 I 


IO 


| use headings to assist reading 


14 


12.8 : 

j 


4.9 


11.9 , 

i 

I ! 


69.6* | 


IO 


. I use visual cues in conjunction with print to make 
! sense of reading 


16 


9.5 


11.6 


: 12.5 , 

i j 


65.4* : 


IO 


i locate author using experience with book elements 


28 


12.0 ! 


11.5 


64.4* 


10.9 ! 


IO 


! locate pertinent details about setting 



* correct answer. 

"The sum of the percentages for each question may be less than 100% because the No Response category is not included. 



Continued 



Category Legend: IO- Identifying the Organization of Information AC-Associatlng/Connecting A-Analyzing SI -Synthesizing Ideas 
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Table 2-8 (continued) 



! 

Item I 


i 

Distribution of Responses (%) 


Category j 


Curriculum Standard 


: 


A i 


B 


c i 


D ! 


; j 


1? 8 i 

i 


3.9 


13.7 \ 


68.5* i 

l 


io • 


attend to typographical features to assist reading 




7.2 S 

! 


1-8 ! 


16.7 


1 

72.6* ! 


KJ 


attend to text features to assist reading 


' 38 I 

; i 


73.9* j 

i 


7.4 ! 

j 


o a I 
O.D 


8.3 


10 


iiqp vi^iial niPQ in finni unfit inn with nrint to makp 

sense of reading 


* . i 

39 


8.2 ! 


72.7* | 


11.8 


5.6 | 


IO 


use headings to assist reading 


] 40 

i I 


12.1 j 


54.2* I 

I 

! 

t 


12.2 I 


19.7 

I 

1 


IO 


use visual cues in conjunction with print to make 
sense of reading 


\ 1 i 


■ 1- 

17.4 j 


34.5 


7.6 


i 

40.0* 


AC 


identify the meaning of a word from its context 


19 ! 
i 


62.0* | 


10.4 


10.8 


13.0 


AC 


decide the most likely meaning if a word using context clues 


i 


20.3 ! 


13.2 


14.2 


51.6* 


AC 


rolsto nortinont HotailQ riPQPrihinn r*h5*i'^r*tpr mntivatinn 
Icldlc pel in id ii vsciciiiO ucounuiiiy uiiciicioici uiuiivcitiuii 


27 | 

j 

! 


8.2 j 


77.9* 


5.3 


7.4 


AC 


decide the most likely meaning of an expression 
using context clues 


\ i 

I 31 i 

1 


5.0 


6.3 


77.5* 


9.3 


AC 


ioentrfy connotations of words beyond their literal meaning 


1 

33 


73.5* 


4.0 


16.1 


4.8 


AC 


identify connotation of a phrase beyond its literal meaning 


i 
i 

34 

i 


26.9 


12.2 


51.3* 


8.0 


AC 


decide the most likely meaning of a word using context clues 


2 


13.7 


79.0* 


4.0 


3.1 


A 


determine explicit relationships between characters 


4 

\ 


75.1* 


6.1 


12.5 


5.8 


! 

A 


recognize the explicit relationship between characters 
and events 


\ 5 


63.8* 


7.9 


10.1 


17.6 


A 


recognize the explicit relationship between characters 
and events 


! 12 

j 


26.6 


4.2 


; 15.9 

I 


52.5* 


A 


determine the relationship between title and passage 


13 

i 


76.7* 


16.0 


i 

3.1 


3.8 


A 


infer character motivation using pertinent details 


15 


3.2 


! 5.1 


87.6* 

j 


3.4 


i A 


infer character's feelings using pertinent detaiis 


18 

i 
j 


j 10.5 

i 
i 


; 3.6 

I 


j 82.1* 

j 
I 


3.3 


i A 


i 

recognize the explicit relationship between characters 
and events 


| 22 


j 17.0 


\ 9.5 


i 

| 4.0 


i 68.6* 


I A 


| determine the relationship among detaiis in the passage 


24 


9.2 


! 15.0 


58.7* 


| 15.7 

I 


A 


recognize key details in the passage 


; 30 


: 5.5 


\ 10.4 


I 

5.2 

I 


i 

i 77.2* 

! 


! A 


infer implicit relationships between characters and events 
; using pertinent details 


'correct answer 










Continued 
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Table 2-8 (continued) 



Item 



Distribution of Responses (%) 



Curriculum Standard 



'correct answer. 
Category Legend: 



1 


A 1 


B | 


c -4 


D ! 


! 


I 


35 ; 


11.9 j 


77.0* 


4.2 ; 

! 


5.3 ; 

! 


— 

A ! 

J 


determine poet's attitude towards subject 


I ! 
! 36 j 

i 


| 

46.7 j 


7.5 


37.2* | 

-4 


i 

6,5 

i 


A j 


infer poet's purpose for writing a poem 


1 -+■ 

t : 

i 3 ! 

1 


2.8 


8,0 ! 


86.1* ! 

I 
i 


i 

2.7 j 

i 


si ! 


judge character motivation using pertinent details 


! 6 i 

, j 


19.0 


14.5 


I 

54.0* ; 

I 


12.0 


SI 

I 


make a judgement about key details in the passage 


1 i 

i 7 1 

1 \ 


6.1 


73.9* 


9.6 

j 


9.8 * 

I 


SI 


draw a conclusion by relating what she/he knows to new 
information in the passage 


! • i 

i 


5.3 


9.1 


70.8* 


14.4 


SI 


make a generalization about characters using 
pertinent details 


i 10 ! 

i 

! : 


81.1* 




5.9 


i 

7,6 


SI 


make a generalization about characters using 
pertinent details 


S 11 

I 


27.0 


4,3 


5.8 


62.3* 


SI 


judge character motivation using pertinent details 


17 


81.8* 


4.0 


2.2 


11.0 


SI 


draw a conclusion by relating what she/he knows to 
new information in the passage 


a, 

j 


15.6 


14.0 


54.1* 


15.3 


SI 


draw a conclusion by relating what she/he knows to 
new information in the passage 


! 23 

j 


49.1* 


7.4 


I 36.5 

j 
i 


5.9 

I 


SI 


draw a conclusion by relating what she/he knows to 
new information in the passage 


I 25 
I 


.9.6 


78.0* 


4.9 

i 
! 


6.2 


SI 


draw a conclusion by using information from the passage 


! 26 

j 


14.9 


69.4* 


4.5 


9.5 


SI 


make a generalization using given information 


j 29 


, 26.5 

I 


40.4* 


j 

• 10.6 


20.7 j SI 

I 


draw a conclusion by relating what she/he knows to 

new information in the passage 

• 
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Examiner's Observations 

Writing Component 

The overall quality of the writing 
in Language Learning in 1993 is 
better than that done in 1989. 
Teachers who marked the tests 
were very pleased with most of the 
papers. They observed specifically 
that students are strong in the 
following areas: planning, story 
structure, and content and 
development. The markers agreed 
that students, for the most part, 
showed genuine confidence in their 



17. What lesson does this story teach 
us? 

81.8 • Be satisfied with the way you 
are. 

4.0 o Be sure to look before you 
leap. 

2.2 o The biggest dog was once a 
puppy. 

1 1 .0 o Donkeys and dogs can be 
friends. 



1. What is the meaning of the 
underlined word seldom ? 

17.4 o Many times 

34.5 o Often 
7.6 o Never 

40.0 • Hardly ever 



writing and hence were willing to 
take the risks that resulted in a 
better product. 

A small number of students wrote 
superior papers. These were lively, 
imaginative, and technically very 
well written. The quality of these 
papers far exceeded the 
expectations for students in a 
Grade 3 classroom. 



Reading Component 

The following is a discussion of 
specific areas of strength and 
weakness for students in Grade 3 
classrooms. Sample questions from 
the test are provided to highlight 
these areas for students achieving 
the acceptable standard and for 
those achieving the standard of 
excellence. For each sample 
question, the filled-in circle shows 
the keyed answer. The percentage 
of students choosing each 
alternative is also provided. 



Acceptable Standard 

For question 17, students had to determine the lesson or moral implied in 
the reading passage.- Those students who achieved the acceptable standard 
were able to do this quite readily. 

In question 1, students achieving the acceptable standard but not the 
standard of excellence had difficulty inferring the meaning of a word from 
its context in a sentence. 



The strengths of students who achieved the acceptable standard include 
an ability to 

• determine the lesson or moral implied in the reading selection (see 
question 17) 

• identify directly stated details of setting and character motivation 

• infer character motivation 

• use visual cues in conjunction with print to make sense of their reading 

However, many of these students did not do as well as expected in 

• recognizing the meaning of words and phrases from the context of the 
material (see question 1) 

• making judgements about, the author's purpose for writing 

• clearly understanding selections that were not the usual story form: 
poems, recipes, and experiments 
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Standard of Excellence 

Students achieving the standard of excellence had no difficulty with the 
following sample questions that accompanied experiments, a written form 
which students achieving the acceptable standard (but not the standard of 
excellence) found more difficult. 

Question 26 required students to synthesize information given in the 
experiment to make a judgement about what people might do. 

Question 7 required students to relate what they know to new 
information presented in the passage and to draw conclusions. 



Clearly, students who achieved the standard of excellence (28.8%) had 
very few difficulties with this assessment. These students could 

• synthesize information from the selection to make judgements/ draw 
conclusions (see question 26) 

• recognize the meanings of words and phrases from the context of the 
material 

• make judgements about the author's purpose for writing 

• make judgements about character motivation 

• understand clearly the selections that were not the usual story form: 
poems, recipes, and experiments 



26. People would MOST LIKELY 
make their own crystal mountains 
because 

14.9 c it is fun to watch them in the 
dark 

69.4 • crystals are pretty and 

interesting 
4.5 o it is good to eat them 
9.5 o crystals are quick and easy to 

grow 



7. The sticks are placed in the 
mixture when it is almost frozen 
so that the 

6.1 o sticks will not break 
73.9 • sticks will stay in place 
9.6 o mixture freezes better 
9.8 o mixture will not separate 



Issues 

During the 1993 marking session, 
teachers were asked to comment 
on both parts of the test. Teachers 
were mostly in favour of the two 
changes that were made in the 
administration of the Language 
Learning Test: the assessment 
was written over two days, the 
writing component on the first day 
and the reading component on the 
second day, and students were 
allowed to discuss the writing 
prompt in groups of two to four. 



Teachers thouf 1 .t that these 
changes were positive because they 
helped students do their best work. 

One area of concern emerged from 
the multiple-choice reading 
component. Questions from the 
catalogue card selection caused 
some misgivings for teachers in 
schools with an automated library 
system. Teachers thought that 
those students familiar with an 
automated system would be 
disadvantaged compared to those 



who have a conventional system in 
their school. In fact, analysis of the 
four questions pertaining to the 
catalogue card indicates that there 
was no advantage to students 
familiar with more conventional 
systems, and on the whole, 
students in the province did very 
well on those four questions (see 
questions 37 to 40). 



17 



Performance-Based 
Assessment 

As part of Alberta Education's 
broadened assessment initiatives, a 
sample of 594 Grade 3 students 
from across the province 
participated in the Language 
Learning Performance-Based 
Assessment (PBA). This activity- 
based assessment using actual 
books was developed by Grade 3 
teachers to assess a broader range 
of reading and writing skills and to 
support classroom instruction. 
Students remained together as a 
class in their regular classroom 



setting, with both their teacher 
and the assessment administrator 
present. By engaging students in 
hands-on book reading followed by 
writing, we hoped to obtain 
valuable information about how 
well students explore, construct, 
and communicate meaning. 

Reading Component 
Genei al Description 

The reading component required 
students to choose a book from a * 
given selection. A range of titles 
was provided, all considered by 
Grade 3 teachers to be appropriate 



for their students. Several titles 
were included for students who 
needed reading material at a 
slightly lower level. After reading, 
students responded to a series of 
open-ended questions. Students 
were given as much time as was 
needed to complete the 
assessment. Copies of the student 
response sheets and scoring guides 
are available from Student 
Evaluation Branch, Alberta 
Education. Provincial results from 
the reading component are shown 
in Table 2-9. 



Table 2-9 

Grade Level 3 Language Learning Performance-Based Assessment 
Percentage Distribution of Scores by Reporting Category 
Reading Component 

June 1993 









Reporting Category 






Score* 


Predictions 


Story Structure 


Retelling Main 
Events 


Characters 


Personal 
Connections 


Opinions 




1.3 


0.5 


5.6 


20 


20 


24 


4 


20.0 


10.9 


10.8 


15.3 


9.1 


. 14.1 


3 

i 


44.1 


44.3 


34.5 


37.9 


37.0 


70.4 


i 

2 


31.8 


33.8 


37.2 


26.6 


30.6 


7.6 


1 


27 


10.4 


10.8 


14.3 


11.1 


32 


0 


0.0 


0.0 


1.2 


3.7 


10.1 


24 



*A score of 3 or above indicates that students have met or exceeded the acceptable standard. Scores within the shaded area (2 or 
below) indicate that students have not met the acceptable standard. 
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Examiner's Observations 

The assessment was divided into six 
scoring categories (Predictions, 
Story Structure, Main Events, 
Characters, Personal Connections, 
and Opinions) and the way in which 
students met the scoring criteria 
varied depending on the category. 

In all, three out of four of students 
achieved the acceptable standard 
for Grade Level 3. Those students 
were able to express predictions 
related to the book cover, 
demonstrate an understanding of 
story events and sequence, and 
identify most of the story's main 
events. They were also able to 
describe the cause of a story 
character's behaviour, make a 
personal connection to what they 
read, and express and loosely 
support an opinion about the book. 
Students who are achieving beyond 
Grade Level 3 expectations were 
able to make predictions based on a 
story problem or theme, describe in 
detail all key features of a given 
story, and express main events of a 
story in a clear and concise manner. 
They were also able to describe 
internal and external character 
motivation, make personal 
connections related to the story 
problem or theme, and supply an 
opinion about the story along with 
justification. 

One quarter of the students did not 
achieve the acceptable standard for 
Grade Level 3. Those students had 
difficulty making predictions 
related to the book. They could 
retell only some story events and 
often included supporting details in 
place of main events. They were 
able to describe what story 
characters did without referring to 
the cause of the behaviours, identify 
but not explain a personal 
connection to the story, and supply 
an opinion about the book without a 
reason. 

O 



In' the category Opinions, a high 
percentage of students, 87%, were 
able to express and support 
opinions at an acceptable standard. 
The category with the next highest 
percentage of students achieving 
an acceptable standard was 
Predictions, where two-thirds of 
the students were able to make 
plausible predictions. Students had 
the most difficulty in the Personal 
Connections category, where they 
were required to connect what they 
read to their personal experiences. 
Only 48% of them achieved an 
acceptable standard. Responses in 
the category Retelling Main Events 
were also weak; only half the 
students achieved an acceptable 
standard. 

These results seem to indicate that 
students achieve at different levels 
when responding to what they 
have read, depending on the 
nature of interaction with the text 
demanded of the students. The two 
categories that had the highest 
percentages of students achieving 
an acceptable standard. Opinions 
and Predictions, are more reader- 
based. That is, success depends 
more on the knowledge and 
background the reader brings to 
the task and is not highly 
constrained by the text. 

The Story Structure and 
Characters categories each had 
55% of students achieving an 
acceptable standard. In order to 
answer these questions at an 
acceptable level, students are 
required to draw on their 
knowledge of the text itself and 
then organize the information. It 
seems that students find questions 
that depend on text-based 
knowledge more difficult than 
those that are primarily reader- 
based. 
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The categories that elicited the 
poorest levels of achievement, 
Retelling Main Events and 
Personal Connections, are those 
that require the students to 
integrate both reader-based and 
text-based knowledge in order to 
make inferences and draw 
conclusions. 

It seems that students do draw on 
their prior knowledge and 
background experience when they 
read. However, they have more 
trouble focusing on the content of 
the text they are reading. They 
have the most difficulty making 
meaning by integrating 
information from their personal 
knowledge and the text. The 
results of the performance-based 
assessment suggest that helping 
students use their own experience 
as well as their knowledge of text 
features will help improve their 
reading comprehension. 

Writing Component 
General Description 

The writing component required 
students to write their own story, 
letter, or series of diary/journal 
entries using an idea that they 
might have generated while 
reading the book during the 
reading component. Students 
brainstormed ideas for writing in 
small groups with others who had 
read the same book. They were 
instructed not to retell the book 
they had read, simply to use an 
event, setting, character, or other 
story features as a springboard for 
their own writing. 

The writing component was scored 
using the same reporting 
categories and criteria as Part A: 
Writing of the Achievement Test. 
Provincial Results are shown in 
Table 2-10. 



Table 2-10 

Grade Level 3 Language Learning Performance-Based Assessment 
Percentage Distribution of Scores by Reporting Category 
Writing Component 

June 1993 



J_. — ... 




Reporting Category 


Standards 




i 

Content and 
Development 


Use of Language 


Conventions 


Meets or Exceeds the Standard of Excellence 


1 


3.7 


3.9 


4.7 


• 

Approaches the Standard of Excellence 


1 

13.9 


13.1 


15.1 


Clearly Meets the Acceptable Standard 


44.6 


434 


39.0 


Approaches the Acceptable Standard 


28.8 




29.8 


Clearly Below the Acceptable Standard 


ai 


as 


105 


INS (Insufficient) 


0.8 


0.8 


0.8 



Examiner's Observations 

Students achieving the acceptable 
standard but not the standard of 
excellence for Grade Level 3 
produced writing that followed a 
format with an identifiable 
beginning, events, and conclusion. 
Events and details were related 
and generally flowed from one to 
another. Endings were fairly 
predictable. Some specific 
vocabulary was used, although 
most words were general in nature. 
Familiar words were spelled 
correctly and unfamiliar words 
were spelled phonetically. End 
punctuation and capitalization 
were usually correct. 

Students achieving the standard of 
excellence for Grade Level 3 
produced writing that contained a 
direct relationship among events, 
providing for a smooth flow from 
one action to the next. Character 
motivation was consistent 
throughout, and most events 
contributed to the main character's 
achievement of his/her goal. These 
writers showed an awareness of 
audience by providing details to 
explain why events occurred and 
by providing information about the 



characters' emotional responses to 
the events. Events were brought to 
an appropriate conclusion. The 
writer's voice was often heard 
through the writing, captivating 
and holding the reader's interest. 
The lengths and types of sentences 
varied. Specific adjectives and 
colourful expressions were found 
throughout the writing. End 
punctuation and capitalization 
were correct. Most words, familiar 
and unfamiliar, were spelled 
conventionally. 

Students below the acceptable 
standard for Grade Level 3 
produced writing that contained 
few events or actions. Beginnings 
were confusing, and subsequent 
events, did not appear to be 
related. Few details were included 
to explain the events, and endings 
were unconnected or non~oxiste.it. 
Words were general and did not 
provide sufficient information to 
the reader. There was little 
evidence that the writer 
understood the correct use of end 
punctuation and capitalization. 
Words were difficult to discern and 
were generally spelled 
phonetically. 



Relationship between the 
Performance-Based Assessment 
and the Achievement Test 

Correlations among all four 
components of the Grade Level 3 
Language Learning assessment 
(multiple-choice reading 
component of the achievement 
test, writing component of the 
achievement test, PBA reading 
component, and PBA writing 
component) were calculated. 
Positive relationships, with 
correlation coefficients ranging 
from .48 to .55, were found to exist 
among all four components. The 
findings of this study are 
confirmed by other research 
studies where correlations average 
about .45. 

This finding is encouraging as it 
indicates that there is a positive 
relationship among the four types 
of Language Learning 
assessments. This would be 
expected if we are, in fact, 
assessing outcomes related to the 
same subject area. The fact that 
the correlation coefficients fall 
within a mid-range is also 
encouraging because it means that 
the performance-based assessment 
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does not exactly duplicate 
information about student 
achievement provided by the 
machine-scorable component. 
Somewhat different aspects of 
achievement are being assessed. We 
are learning something different 
about what students know and can 
do from each of the two types of 
assessment. 

Performance-based assessment 
allows us to see more closely the 
kinds of knowledge students use 
when they are reading in situations 
similar to those encountered in the 
classroom on a daily basis. In 
responding to open-ended questions, 
students are able to select from a 
variety of strategies, depending on 
their own strengths as well as on 
the demands of the task itself. The 
examination and analysis of those 
responses and the strategies used 
give us more of a profile of the 
processes students use while 
reading. That information can then 
be used to outline more specifically 
students' strengths and weaknesses 
in order to design programs that 
better meet their needs. 



Students did not do as well on the 
writing component of the 
performance -based assessment as 
they did on the writing component 
of the achievement test. Fewer 
students achieved an acceptable 
standard on the performance-based 
assessment in all three reporting 
categories. In Use of Language, 
fully 10% fewer students (60.4% as 
compared to 71.3%) met the 
acceptable standard on the 
performance-based assessment. 

Students had more difficulty with a 
writing task that was connected to 
a reading experience than they did 
on a task that was more open in 
nature. Again, it seems that the 
more students are required to 
integrate a variety of strategies in 
order to complete a task, the more 
difficulty they have. The writing 
component of the performance - 
based assessment required 
students not only to comprehend 
the story they read, but to 
integrate that with their own ideas 
to create an original story. In 
contrast, the writing component of 
the achievement test required 



students to look at a visual and then 
expand on their own ideas. Although 
reading and writing are often 
integrated in Grade Level 3 
classrooms, students have not yet 
mastered the application of a variety 
of strategies associated with reading 
and writing to a single task. 

These findings support the idea that 
a combination of assessment 
.strategies (multiple-choice, 
performance-based assessments, 
and ongoing classroom-based 
assessments) are needed to provide 
a complete picture of student 
achievement in Language Learning. 

Context for Learning 
General Description 

In June 1993, a random selection of 
1 961 Language Learning students 
in Grade 3 participated in a study 
that surveyed students' attitudes on 
various language learning issues. 
The results from the survey are 
presented in Table 2-11. 



Table 2 11 

Grade Level 3 Language Learning 

Survey on Attitudes Toward Language Learning 

Percentage Distribution of Student Responses 

June 1993 



Statement 

1 . I think reading is an important skill. 

2. Everyone needs to know how to read. 

a After I've read something by myself, I like to: 

a Draw a picture 

b. Tell someone about it 

c. Find another book like it 
d Write something about it 



Yes 

95.5 
83.6 



46.0 
47.0 
51.5 
35.0 



No 

1.3 

7.4 



35.5 
33.1 
28.9 
43.3 



Not Sure 

29 
8.5 



1Z6 
12.7 
12.9 
13.1 



No Response 

0.4 . 
0.5 



5.9 
7.1 
6.8 
8.6 
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Table 2-11 (continued) 



Statement 


I 

Yes 


No 


Not Sure 

1 


No Response 

i 


i 4. 

1 


l am a good reader. 


74 9 


R R i 


18.8 | 


1.4 


5. 


I am a good writer. 


fift 1 
DO. 1 




91 ft 


16 i 


a 


I like writing: 












a Reports 


46.4 


345 


13.0 


6.1 


i 


b. Journals/diaries 


473 


32.7 


13.1 


6.9 




c. Letters 


65.9 


17.2 


115 


.5.7 




d Stories 




8.3 


7.5 


42 




e. Poems 


CO Q 


oc c 


1 sj.l 


O.C7 


Statement 


Often 


Sometimes 


Never 


No Response 


7. 


I keep my writing in a writing folder. 


42.3 


40.1 


145 


3.0 


a 


Before I write, I like to: 










i 
i 


a Talk to my classmates 


27.8 


44.5 


ppp 


5.4 




b. Draw a web 


223 


33.0 


40.4 


4.3 




c. Make a chart 


21.4 


372 


35.8 


5.6 




d. Listen to my teacher 


64.5 


23.3 


7.8 


4.4 




e. Imagine 


62.6. 


24.9 


9.4 


3.1 


9. 


I like to: 












a Talk about books I have read 


29.2 


44.7 


21.3 


4.8 




b. Listen to my teacher read stories 


74.4 


18.5 


4.0 


3.1 




a Read during my spare time 


47.6 


40.0 


8.3 


4.0 


i 


d. Write during my spare time 


25.9 


48.7 


20.9 


4.6 


10. 


I choose my books based on: 






i 






a Author 


245 


j 38.5 


! 327 


4.6 




b. Topic (for example, mysteries) 


56.8 


28.8 


10.4 


! 4.1 




a The pictures t 


df.O 


OO.0 




O.l 


i 


d How hard the words are 


34.4 


35.3 


I 25.9 

I 


j 4.5 

i 


j 11. 


I use the school library to choose: 




j 

i 
i 


! 






a Stories for free reading 


j 59.6 


i 

28.7 


i 

; 81 


i 

i 37 


j 


b. Information books (for example, animal books, 


39.5 

l 


41.9 


! 13.4 

I 


; 5.3 




books on amazing facts, or encyclopedias) 


i 




i 






c. Books to learn how to do things 


! 36.6 


j 46.7 

I 


j 12.1 


4.5 



Continued 
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Table 2-11 (continued) 







Statement 


Often 


: * l 
: 

i Sometimes 

A 


Never 


I 

No Response | 


z 


This year, I read or used: 




• 




1 

i 




a 


Magazines 


27.0 


39.8 


27.8 


5.5 




b. 


Newspapers 


19.3 


t 

! 32.5 


4Z8 


5.4 




c. 


Dictionaries 


37.9 


j 43.1 


14.1 


4.8 




d. 


encyclopedias 


23.0 


j 39.6 


32.3 


5.0 




e. 


Atlas/Globe 


273 


j 41.8 


25.8 


5.1 




< 

i. 


Poetry 


36.5 


\ 39.6 


18.5 


5.5 




9- 


Fiction books 


59.9 


! 26.7 


8.7 


a j 




h. 


Information books (non-fiction) 


4Z6 


| 41.3 


10.4 


5.8 




i. 


Comics 


53.1 


I 26.6 

i 


16.0 


4.3 



Student Attitude 
Questionnaire 

Results indicate that of the 
students surveyed, 95.5% believe 
that reading is an important skill. 
Analysis of the student 
questionnaires indicate that the 
majority of students (85.6%) often 
or sometimes choose their reading 
material by topic and the most 
often read materials are fiction 
books, comics, and information 
books. Students use the school 
library mainly for choosing leisure 
reading materials. Students' 
favoured types of writing are 
stories, letters, and poems; 74.4% of 
the surveyed students like to listen 
to their teacher read stories. 

The students surveyed tended to 
underestimate their abilities in 
writing and reading; 68.1% of these 
students thought they were good 
writers, and 74.2% thought that 
they were good readers. In actual 
fact, on the achievement test 
81.3% and 82.2% of students 
achieved the acceptable standard in 
writing and reading respectively. 

Relationship between Student 
Attitudes and the 
Achievement Test 

We were able to compare the 
results of 940 of the 1 961 students 



who participated in the attitude 
survey with their Grade Level 3 
Language Learning Achievement 
Test results. Of the 152 students 
who achieved the standard of 
excellence on the total achievement 
test, 90.1% believe that they are 
good readers. This is sharply 
contrasted by students who 
performed below the acceptable 
standard (?i=132), where only 
44.9% believed themselves to be 
good readers. 

Students achieving the standard of 
excellence on the total test tend to 
be capable and enthusiastic 
readers. It was not surprising to 
find that 68.5% of those students 
achieving V standard of excellence 
read for enjoyment during their 
leisure time, whereas the same is 
true for only 46.2% of students who 
only achieved the acceptable 
standard and 40.8% of those 
performing below the acceptable 
standard. 

It is interesting to note that when 
students who are below the 
acceptable standard read in their 
spare time, they do so in order to 
learn something specific more often 
(40.2%) than do students who 
achieved the acceptable standard 
(38.8%) or who achieved the 



standard of excellence (30.2%). This 
reinforces the belief held by many 
teachers that reading, especially for 
weaker students, must have a real 
purpose in order for students to 
actively engage it. 

In preparation for writing, only one 
in four students surveyed like to talk 
to classmates, draw a web, or make 
a chart, whereas two out of three 
students in this group like to listen 
to their teacher and imagine. These 
activities are quite different in 
nature; the former three require the 
student to be actively involved with 
others or alone when producing a 
product, whereas the latter two are 
activities that allow students to be 
more passive. This suggests that 
students are more comfortable in 
situations where they are required 
to take less personal risk. 

Teacher Questionnaire 

In July 1993, 280 teachers from 
across the province, including 150 
teachers who were marking the 
writing component of the Grade 
Level 3 Achievement Test, 
participated in a study that 
surveyed the instructional strategies 
and learning environments they 
provided in the classroom. The 
results from the survey are 
presented in Table 2-12. The vast 
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majority of the teachers surveyed 
provide time for silent reading at 
least once per week. Teachers report 
that classroom environments are 
"print rich" with magazines, 



newspapers, both fiction and non- 
fiction books, as well as reference 
materials. Students who achieved 
the acceptable standard tend to 
have access to these resources 



slightly more often than do students 
who have not yet achieved the 
acceptable standard. It is interesting 
to note that all teachers surveyed set 
aside time to read to their students. 



Table 2-12 

Grade Level 3 Language Learning 

Teacher Survey on Instructional Practices and Learning Environments 
Percentage Distribution of Teacher Responses 

June 1993 



Statement 



Never 



: 1. Did your students write in LL? 

i 

j 2 Did your students write in a journal for LL? 

j 3. Did your students use the school Library? 

4. Did you set aside time to read to your students? 

j 5. Did your students do silent reading? 



Statement 



10. 



11. 



Did your students have a choice about what they 
wrote? 

Did your students do prewriting activities before 
they wrote? 

Did your students use a computer at school to do 
the following? 

a All of their writing (from draft to final copy) 
b. Some of their writing 

Did your students write a number of drafts before 
handing in the final copy? 

Did you have conferences with your students 
about their writing? 

Did you use the following instructional strategies 
with your students? 



0.0 
4.3 
0.4 
0.0 
0.0 



Loss Than 
Once a 
Week 



Once a 
Week 



\ Several 
j Times a 
I Week 



Every Day 



No 
Response 



1.8 
12.1 
0.7 j 



I 



Never 



0.0 



0.0 



25 
0.7 



7.5 
27.1 
61.8 
3.9 
3.9 



Rarely Sometimes 



46.1 
40.7 
32.1 
36.4 
25.0 



42.5 
14.3 
3.9 
56.4 
70.0 



Often 



0.7 



32 



21.4 



21.8 



59.6 



! 43.9 



Always 



17.5 



30.4 



21 

1.4 
1.1 
0.7 
0.4 



No 
Response 



0.7 



0.7 



63.6 
46.8 

4.6 



15.7 
232 

23.9 



1.1 ! 13.9 



10.7 
21.1 

33.6 



36.8 



1.8 ! 
6.1 j 

225 ! 

j 

34.6 I 



0.7 
0.4 

11.4 



9.6 



7.5 
25 

3.9 



3.9 




a 


Reading workshop 


20.7 I 


132 


22.9 


23.9 


7.5 


I 11.8 


b. 


Paired Reading 


1 29! 


11.8 


282 1 


45.4 1 


8.6 


i 32 


c. 


Learning Centres 


129 : 


24.3 


29.3 


23.6 


4.6 


! 5.4 


d. 


Drama 


: 4.6 ! 


39.3 


37.1 


13.6 i 


1.1 


; 4.3 


e. 


Group Work 


0.7 ; 


3.2 


23.6 


57.9 


12.5 


: 21 


f. 


Cooperative Learning 


4.3 


11.4 


22.9 


432 : 


15.0 


















Continued 
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Table 212 (continued) 



Statement 



No 



\3S 



No Response 



j 12. Did you use the following types of assessments with your 
; students? 



13. 



14. 



15. 



a 

b. 
c. 
d 



9- 
h. 



Multiple-choice test 
Short-answer test 
True-false test 

Interpretation of charts, diagrams, pictures, 
graphs, and maps. 

Creation of a diagram, picture, mural, collage, or 
model to show understanding 
Oral presentation 
Arrangement of events >n order 
Classification of pictures/words/phrases 



j 243 

j 11.1 

j 49.3 

I 1.8 



3.9 

3.9 
6.8 
8.9 



Were the following materials available in the classroom 
for the students? 



70.0 
82.9 
43.6 
91.8 

90.0 

90.7 
87.5 
84.3 



5.7 
6.1 
7.1 
6.4 

6.1 

5.4 
5.7 
6.8 



16. 



a Magazines 


9.6 


84.6 


5.7 


b. Newspapers 


50.4 


42.1 ! 


7.5 


c. Dictionaries 


0.7 


943 


5.0 


d Encyclopedias 


41.4 


50.7 


7.9 


e. Atlas/Globe 


7.1 


87.9 


5.0 


f. Poetry 


3.2 


91.1 


5.7 


g. Fiction books 


0.0 


95.0 


5.0 


h. Non-fiction books 


1.1 


93.6 

I 


5.4 


Are the following reading series available in your school 




i 




for you to use? 




i 

i 




a Early Bird Collection 


673 


! 

6.1 


26.1 


b. Impressions 


18.2 


| 70.0 


11.8 


c. Journeys 


45.4 


| 32.5 


22.1 


d Networks 


| 35.7 


! 43.6 


20.7 


e. Unicom 


| 59.3 


| 11.8 


28.9 


f. Waves 


66.8 


! 3.6 


29.6 


g. Reflections 


j 65.0 


| 5.0 


30.0 


Did your students, as a class, experience the following? 




j 

i 




a Live theater 






I 


b. Concert 


I 21.4 


! 71.8 


j 6.8 


c. Symphony/Opera 


\ 24.3 


! 67.1 


8.6 


d. Movie 


| 62.1 


j 26.8 


11.1 


e. Museum/Art Gallery 


9.6 


84.6 


i 5.7 




j 34.6 


I 56.8 


i 8.6 


Did your students have a portfolio or writing folder for 


j 




I 


samples of iheir writing? 


8.9 


85.4 


; 57 



9 



38 



Slightly more than one in four 
teachers surveyed said that their 
students use the computer for 
some of their writing at least once 
per week. It is an encouraging sign 
that the technology is becoming 
more readily accessible in the 
lower elementary grades; as it 
becomes more available, we should 
see an emerging trend of younger 
students actively using the 
technology for more of their work. 

Of the teachers surveyed, one in 
four did not respond to the 
question about the reading 
resources in their schools. They 
appear to be uncertain about the 
availability of the specific 
resources mentioned in this 
survey. 

Relationship between 
Learning Environment and 
the Achievement Test 

Achievement test results of 509 
students were matched with 



teachers who participated in the 
survey of instructional practices 
and learning environments. 
Students were split into three 
groups: students achieving the 
standard of excellence , students 
achieving the acceptable standard 
but not the standard of excellence, 
and students not yet achieving the 
acceptable standard. Differences in 
reported teaching behaviours 
matched to these students were 
examined. All of the teachers in 
this matched study reported that 
their students used their school 
library at least once per week. All 
teachers provided field trip 
experiences outside the regular 
classroom milieu: 92.8% of 
teachers of students who achieved 
the standard Gf excellence provided 
an opportunity for viewing live 
theatre, and 100% of these 
teachers took their students to art 
galleries and museums. 



The study showed that 25.5% of 
teachers of students achieving the 
standard of excellence always have 
their students write multiple 
drafts of compositions, compared to 
18.0% of teachers of students 
achieving the acceptable standard 
and 15.3% of teachers of students 
who were below the acceptable 
standard. 

Furthermore, 7.2% of the teachers 
of students who achieved the 
standard of excellence always 
conference with their students 
about their writing, whereas 6.8% 
of teachers of students achieving 
the acceptable standard and only 
4.7% of those teachers of students 
who werfe below the acceptable 
standard follow the same practice. 
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Section 3 

(irade 6 
Social Studies 

v 



In 1993 a number of assessments 
were carried out in Grade 6 Social 
Studies. The achievement test was 
administered to students province- 
wide. Student self-reports and 
teacher ratings were collected in a 
sample of classrooms as a means of 
gathering information about 
process skills. As well, a sample of 
Grade 6 Social Studies studen' 
and teachers completed 
questionnaires designed to 
examine the relationships among 
various contexts for learning and 
achievement. The results from all 
these assessments follow. 



Achievement Test 
General Description 

The Grade 6 Social Studies 
Achievement Test had two parts. 
Part A consisted of 50 multiple- 
choice questions and Part B was a 
writing assignment. Part A was 
worth 70% and Part B was worth 
30% of the total test mark. 
Students had 60 minutes to 
complete each part of the test. 

Statistics for the total test and for 
the components are based on the 
results achieved by 32 630 
students: 29 802 wrote the regular 
form and 2 828 participated in the 
achievement-over-time study, as 
shown in Table 3-3. This section of 
the report provides answers to the 
following questions: 



• How many Grade 6 students 
wrote each form of the test or 
were absent or exempt? 

• What percentage of students who 
wrote the Grade 6 Social Studies 
Achievement Test achieved the 
acceptable standard? 

• What percentage of students who 
wrote the Grade 6 Social Studies 
Achievement Test achieved the 
standard of excellence ? 

• What did Grade 6 students know 
and what could they do in Social 
Studies? 

• What parts of the Social Studies 
curriculum were difficult for 
Grade 6 students? 



Summary of Results 

Results show that 80.8% of the 
students who wrote the test 
achieved the acceptable standard 
and 8.0% achieved the standard of 
excellence. These results were 
lower than expected. 



Content of the Test 

Of the 50 multiple-choice questions 
in Part A, 19 questions on Topic A, 
which dealt with Local 
Government, 15 were on Topic B, 
which dealt with Greece: An 
Ancient Civilization, and 16 were 
on Topic C, which dealt with 
China: A Pacific Rim Nation. 



The Part B writing assignment 
asked students to wrile one to 
three paragraphs on the issue 
"Should governments help people 
meet their needs?" 

The test assessed students' 
knowledge of facts, concepts, and 
generalizations, as well as their 
application of process skills such a. 
using information to draw 
conclusions. 
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Test Blueprint 

The test blueprint for Part A: 
Multiple Choice shows the 
distribution of questions according 
to the curricular content area 



(topic) being assessed and 
according to the knowledge and 
process skills required to answer 
the question. 



Table 3-1 

Grade 6 Social Studies 
Achievement Test Blueprint 
Part A: Multiple Choice 

June 1993 







Question Distribution 






Knowledge/Skills 


Topic A 
Local Government 


Topic B 
Greece: An Ancient 
Civilization 


Topic C 
China: A Pacific Rim 
Nation 


Proportion of 
Total Score 


owiedge 


Understands generalizations, 
concepts, related facts and 
content 


1,2,3,18 
22,36,37,39,49 


4, 9,10,11, 

17, 20, 21,25,27 


8,16, 30,31, 
32, 33,34,35 


36% 


c 












Skills 


Locating, organizing, interpreting 
information* 


40, 42, 46, 47, 48 

___ 


7, 26,44 


13, 14, 28, 29, 
43,45 




Process ! 


Analyzing, synthesizing, 
evak^ting 


19, 23, 38,41,50 


5, 6,24 


12,15 


34% 




Proportion of Total Score 


27% 


21% 


22% 


70% 



* This includes the section in the Program of Studies entitled "Geography/Mapping" under Process Skills. 
Three questions were deleted because the choices were ambiguously worded (22, 33, 45). 
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The blueprint for Part B: Written 
Response shows the reporting 
categories and the proportion of 
score. 

Table 3 2 

Grade 6 Social Studies 
Achievement Test Blueprint 
Part B: Written Response 

June 1993 



Description of Writing Activity 

1 


Reporting Category 


Proportion of Score 


Express an opinion/position using 
examples for support. 


1. State and Support an Opinion/Position 

Students are expected to persuasively give their 
opinion/position and use relevant examples to 
support it. 


20% 


l 
i 

i 

i 

1 


2. Quality of Language and Expression 

Students are expected to express their ideas in an 
organized manner with clear, effective language, 
using correct conventions of spelling, punctuation, 
and grammar. 


10% 


i 

: 
1 


TOTAL 


30% 
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Student Participation 

In June 1993, principals reported a 
total population of 39 097 students 
in Grade 6. Table 3-3 presents the 
number and percentage 
distribution of students who wrote 
the Grade 6 Social Studies 



Achievement Test, who were 
absent, or who were exempted. In 
total, 89.7% of the students 
reported to be in Grade 6 in June 
1993 wrote the Grade 6 Social 
Studies Achievement Test. Results 
for Francophone and French 



Immersion students are reported 
directly to participating schools 
and jurisdictions and are not 
included in the calculation of the 
provincial results presented in this 
section of the report. 



Table 3-3 

Grade 6 Social Studies 
Student Participation 

June 1993 



■ " 1 

Category 


Number 
of Students 


Percentage 
of Students 


total NuniDer ot oiuasms m negutar rTograms 


no Ron 


Oo.D 


Students Who Wrote the Regular Form 


29 802 


76.2 


Students Who Wrote the Achievement-Over-Time Forms: 
Form F (Golden rod)— Same as the Regular Form 
Form E (Cherry) 
Form D (Grey) 


2 828 
961 
932 
935 


7.2 
2.5 
2.4 
2.4 


Other Students Who Wrote: 

Francophone— Taught in French, Wrote in French 
Francophone— Taught in French, Wrote in English 
French Immersion— Taught in French, Wrote in English 
French Immersion— Taught in French, Wrote in French 


2 432 

0 

176 
24 
2 232 


6.2 

0.0 
0.5 

°< 

5.7 


Students Absent 


1 709 


j 3.2 


Students Exempted from Writing 


2 326 


I 14.7 


j Categories of Exemption: 






; 1 . Special Needs Students 

j 2. Subject Was Not Taught This Term 

| 3. Students in English as a Second Language 

i 4. Language of Instruction Was Not English 

| 5. Other (as approved by the Superintendent) 


1 072 
508 
407 
63 

j 276 


j 2.7 
| 1.3 

! 10 

; 0.2 
i 07 


I Total Principals' Reported Population 
I Test Day, June 1993: 


! 39 097 


! 100.0 



Grade 6 Enrollment: September 30, 1992 



38 484 



* Provincial results are based on test scores achieved by students who were instructed in English and who wrote the English form of the 
achievement test, and those students who were instructed in a language other than English or French and who wrote the English form of 
the test. They are indicated by the shaded area of the table. 
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Figure 3-1 

Grade 6 Social Studies 

Percentage of Students Achieving Acceptable Standard on the 
Total Test and on Components of the Test 

June 1993 




Knowledge 



Process Skills 



Total Test 



* Provincial Achievement Standard— 85% of the students who wrote the test are expected to 
achieve the acceptable standard 



Figure 3-2 

Grade 6 Social Studies 

Percentage of Students Achieving Standard of Excellence on the 
Total Test and on Components of the Test 

June 1993 




Knowledge 



Process Skills 



Total Test 



' Provincial Achievement Standard— 15% of the students who wrote the test are expected to 
achieve the standard of excellence 



Results in Relation 
to Standards 

Achievement test results are 
reported in relation to assessment 
and achievement standards. The 
Provincial Assessment 
Standard is the lowest score on a 
test that a student must achieve for 
his/her performance to be judged 
"acceptable" and/or "excellent". The 
Provincial Achievement 
Standard refers to the percentage 
of students writing the test who are 
expected to achieve the Provincial 
Assessment Standard. 

Figures 3-1 and 3-2 and Tnble 3-4 
show the percentage of students 
achieving provincial assessment 
standards in relation to provincial 
achievement standards on the total 
test and on components of the test. 
These data are based on the results 
of the 32 630 students in regular 
programs who wrote the test. They 
show that the percentage of 
students achieving the standard of 
excellence on the knowledge 
questions was considerably higher 
than expected. Fewer students than 
expected achieved all other 
standards set for the Grade 6 Social 
Studies Achievement Test. 

When interpreting these findings, it 
is important to remember that they 
are based on the test scores of only 
a portion (83.5%) of the students 
reported to have been enrolled in 
Grade 6 in June 1993, i.e., those 
students in regular programs who 
wrote the achievement test. These 
figures tell us nothing about the 
levels of achievement of students in 
Francophone and French 
Immersion programs, of students 
who were absent on the day of 
writing, or of those students who 
were exempted from writing. 



Earlier studies on the language of 
testing show that the standards 
developed for the English version 
of the test cannot be directly 
applied to French Immersion 
students writing the translation; 
standards for the French 



Immersion and Francophone 
students were not set separately 
due to the small numbers involved. 
Thus, these students are not 
included in any discussion of the 
numbers of students achieving 
standards. 
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Table 3-4 

Grade 6 Social Studies 

Students Achieving Standards on the Total Test and on Components of the Test 

June 1993 



... 














: 




Provincial 


Provincial 


Students Achieving Assessment 


t 


Maximum 


Assessment 


Achievement 




Standard 






Possible 


Standard* 


Standard" 


Expected 


Actual 


Actual 


Reporting Category 


Score 


(Raw Score) 


(Percent) 


Number 


Number 


Percent 


( 1 

1 Standard of Excellence 














i Total Test 


100 


80 


15 


4 895 


2 621 


8.0 


! Skills Only 


64 


50 


15 


4 895 


2 015 


6.2 


j Knowledge Only 


24 


20 


15 


4 895 


9 352 


28.7 


j Acceptable Standard 














Total Test 


100 


49 


85 


27 736 


26 368 


80.8 


Skills Only 


64 


30 


85 


27 736 


25 195 


77.2 


Knowledge Only 


24 


13 


85 


27 736 


27 249 


83.5 


Below Acceptable Standard 


N/A 


N/A 


N/A 


N/A 


3 483 


10.7 


on Both Components 















*The Provincial Assessment Standard is a score determined by standard-setting procedures and is the lowest score a student must 
achieve for his/her performance to be judged "acceptable" and/or "excellent" in relation to curricular expectations. See Appendix A. 



The Provincial Achievement Standard refers to the percentage of students expected to achieve the Provincial Assessment Standard. 



It should be noted that the actual 
percentages of students achieving 
standards on the total test for 
Grade 6 Social Studies (8.0% and 
80.8%) are based on 32 630 
students in the regular programs 
who wrote the test. 

If the percentages are based on the 
total June population reported by 
principals (not including the 
French Immersion and 
Francophone students who wrote 
the test), the percentages achieving 
standards would be: 

7. 1% achieving the standard of 

excellence, 

71.9% achieving the acceptable 
standard. 



If the percentages are based on the 
September 30, 1992, Grade 6 
enrollment (36 052) not including 
the Francophone and French 
Immersion students who wrote the 
test, the percentages achieving 
standards would be: 

7.3% achieving the standard of 

excellence, 

73.1% achieving the acceptable 
standard. 

It is emphasized that the 
percentages based on enrollment 
present the lowest estimate of 
achievement. It is highly likely . 
that some of the students who were 
absent, exempt, or not accounted 
for could have achieved standards. 
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The number of students achieving 
the acceptable standard and the 
standard of excellence for each 
school was analyzed to determine 
whether schools were below, 
meeting, or exceeding provincial 
achievement standards. Schools 
classified as meeting provincial 
achievement standards were those 
for which the difference between 
the actual number of students and 
the expected number of students at 
or above standards was not 



statistically significant. Differences 
are only reported when there is a 
5% or smaller probability that a 
difference of that size could occur 
by chance. The results are reported 
in Table 3-5. 

Table 3-5 shows the 81.2% of 
schools were considered to have 
achieved the provincial 
achievement standard for the total 
test at the standard of excellence. 
About one-quarter of schools were 



significantly below the provincial 
achievement standard. School 
administrators and teachers 
should critically examine social 
studies achievement test results to 
determine what they are going to 
do to improve the level of student 
achievement. Although this is true 
for all schools, it is particularly 
important for those schools 
performing below the provincial 
achievement standard. 



Table 3-5 

Grade 6 Social Studies 

Percentage Distribution of Schools* Meeting Provincial Achievement Standards on the Total Test 
(N = 993) 
June 1993 



Standard 


Significantly Below \ Not Significantly Different 
Provincial Achievement ! From Provincial 
Standard j Achievement Standard 


Significantly Above 
Provincial Achievement 
Standard 


. . — . — 

Standard of Excellence 

Acceptable Standard 

i 


18.8 ' 78.7 

! 

26.4 1 67.5 

1 


2.5 
6.1 

_ 



'Schools with fewer than five students are excluded, as the statistical significance of the difference between the number actually meeting the 
standard and the number expected to meet the standard when calculated and reported is not educationally meaningful. 



Results for Part A: 
Individual Multiple- 
Choice Questions 

Tables 3-6A to 3-6C show the 
percentage of students who chose 
each alternative (A, B, C, and D) 
for each multiple-choice question. 

The results shown in these tables 
can best be used in conjunction 
with the tables in jurisdiction and 



school reports. Variations in 
patterns of students' responses to 
questions can help to indicate 
strengths and weaknesses in local 
educational programs. 

Statistics represented in these 
tables are based on results 
achieved by 30 763 students (those 
who wrote the regular form and 
those who wrote Form F). 



The sum of the percentages for 
each question may be less than 
100% because the No Response 
category is not reported in these 
tables. 

For a discussion of individual 
multiple-choice questions, see 
Examiner's Observations, pages 38 
to 40. 



ERIC 
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Table 3-6A 

Grade 6 Social Studies 

Results for Individual Multiple-Choice Questions 
Topic A — Local Government 

June 1993 



; 

Item h- 


i 

Distribution of Responses (%) 


I 

Component 


Category 


1 

Curriculum Standard 


i 


i 

A | 


B ! 


C 


D 


; 1 i 


i 

8.2 ! 


17.2 | 

i 


72.5* 


I 

2.1 | 


Knowledge 


C 


know examples of physical needs 


i > 

: 2 : 

I 


30.0 j 


39.3* 


9.6 | 


! 

20.7 


Knowledge 


c 


know examples of social needs 


1 ! 

3 ! 

] : 


1 

76.1* j 

! 


3.7 


10.3 

I 

| 


9.8 


Knowledge 


c 


j 

know examples of government services 1 


1 1 

! i 

1 


! 

14.1 


10.5 


60.2* 


15.1 


iMIUWIcUyc 

. -\ 


c 

1 , 


itnHors l anH tho rinht nf frPP QDPPPh 


| 

18 ■ 


5.2 


9.0 


76.9* 


8.8 


Knowledge 


G 


understand the democratic process 


1 37 1 

1 i 

1 


6.6 


10.5 


8.2 


74.6* 


Knowledge 


G 


understand the responsibilities of a citizen 
in a democracy 


1 j 

! 39 | 


17.3 


7.1 


1.4 


74.1* 


Knowledge 


G 


identify a lobby group activity 


! „ 1 

i 49 ! 


12.7 


16.7 


30.0* 


40.2* 


Knowledge 


G 


understand appropriate lobby group activity 


40 ! 


12.0 


61.2* 


9.2 


17.1 


Process Skills 


L 


compare information on a graph to a chart 


I 42 ] 

I \ 


18.3 


21.1 


51.6* 


8.9 


Process Skiils 


L 


interpret information to distinguish 
fact from opinion 


i ; 
; 46 


11.5 


57.1* 


28.3 


2.8 


Process Skills 


L 


use scale to calculate distance 


1 1 
! 47 ! 


22.9 


17.7 


35.4* 


23.6 


i Process Skills 


1 L 


interpret maps to discover relationships 
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i 19 



87.5* 



5.6 



3.7 



23 72.3* 

38 j 11.4 

41 j 25.3 

50 1 13.2 

! 



47.4* | 11.3 i 21.7 j 

! | i 

18.4 



11.7 
36.5* 

26.4 



3.4 
70.9* 
17.0 
30.6* 



2.8 

19.3 
6.0 
5.7 
20.9 
29.3 



Process Skills 

Process Skills 
Process Skills 
Process Skills 
Process Skills ' 
Process Skills 



A 
A 
A 
A 
A 



of scale 

select an efficient means of communicating 
information 

draw a conclusion about democracy 

draw a conclusion based on given information 

analyze the result of actions in a democracy 

judge how useful the data in a chart would be 
for different purposes 

evaluate sources of information 



*correct answer (Item 49 has two correct answers) 

Category Legend: F— Facts 

C — Concepts 

G— Generalizations 



L— Locating, Organizing, Interpreting Information 
A— Analyzing, Synthesizing, Evaluating 



Continued 
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Table 3-6B 

Grade 6 Social Studies 

Results for Individual Multiple-Choice Questions 
Topic B — Greece: An Ancient Civilization 

June 1993 



j \ 
item f 


Distribution of Responses (%) 


- T 


Cateaorv I 

1 
• 


"1 

1 

Curriculum Standard ! 

i 


A 


B ! 


C I 


0 j 


A ' 

i 4 • 


r 

7.1 j 


1 

1 

14.2 ! 


h 

5.0 | 


73.5* 


Knowlpdnp 


h 

j 

F 

i 


i 

understand psychological needs j 


i ; 

9 ! 


53.9* j 


14.1 


25.2* | 

i 
I 


6.7 


! 

Knowledge 


i 

F j 

j 


i 

identify a psychological need in Ancient ; 
Greece j 

i 


1 10 

! 


2.5 ] 


4.6 


1.2 | 


91.7* 


Knowlpdnp 

i 

1 

j 


F ! 

i 

l 

j 


! 

understand the role of slaves in Ancient 
Greece 


j 

11 

! 


56.6* 

: 


7.9 


I 

14.3 

i 


21.2 


1 

Knowledge 


S 

F | 


know how Ancient Greeks met some 
physical needs I 

t 


! 17 


: 

13.9 

| 


10.7 


10.6 

• i 

; i 


64.7* 


Knowledge 

| 


F. 


understand the role of the Assembly in 
Ancient Greece 


j 20 

i 


| 8,3 

j 
I 


80.7* 


I S 7 

! 

i 

I 


5.0 


| Knowledge 

! 
i 

! 


F 


understand that democracy is common to 
Canada and Ancient Greece 


| 27 


93.6* 


2.4 


! 1.8 


2.2 


! Knowledge 

! 
« 


F 


understand the role of slaves in 
Ancient Greece 
















I 21 


! 11.2 

I 


59.4* 


I 17.4 


11.9 


1 Knowledge 

i 


G 


understand the purpose of government 


i 

| 25 


i 

j 10.6 


| 20.5 

i 


| 66.0* 


2.7 


knnu/lorlnp 
rxllUWlctiyt/ 


G 


1 

understand class structure in Ancient Greece j 

i_ .. . { 


7 


j 36.6* 

! 


; 19.0 


| 33 1 

i 


! 11.1 


| Process Skills 


; L 
1 


i 

interpret comments to identify an opinion j 


i 26 


j 

I 25.6 

] 


i 

j 32.6 


J 33.6* 

i 


8.0 


| Process Skills 


i L 


1 1 
| interpret comments to identify a fact j 

1 1 


; 44 


j 22.6 

i 


| 25.4 


S 20.3 

J 


I 

31.3* 

j 
i 


t Process Skills 


i l 

i 

1 


■ i 
| use latitude and longitude to locate i 
geographical places 


I 5 


| 8.3 

I 


{ 16.7 




j 68.7* 

i 


j Process Skills 


A 

i 


synthesize information to select the 
best conclusion 


i 

I 


i 8.7 

i 


j 

77.6* 

i 


s 


| 8.7 

i 


! Process Skills 

i 


A 

i 
« 


draw a conclusion based on various 

i comments 

I 


• 24 


5.3 


I 86.4* 


j 

i 6.8 


i 

! 1.4 


Process Skills 


i a 


j 

; evaluate statements of opinion 



•correct answer (Item 9 has two correct answers) 

Category Legend: F— Facts 

C— Concepts 

G —Generalizations 



L— Locating, Organizing, Interpreting Information 
A— Analyzing, Synthesizing, Evaluating 



Continued 
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Table 3 6C 

Grade 6 Social Studies 

Results for Individual Multiple-Choice Questions 
Topic C — China: A Pacific Rim Nation 

June 1993 



! ! 

j : Distribution of Responses (%) 


Component 


Category 


■ 


: item | 


A ] 


B ! 


C 


D 


Curriculum Standard 


i ! 

! 8 ! 

1 1 


I 

12.1 

I 


3.1 | 


81.8* 


2.9 


Knowledge 


F 


understand role of grandparents in meeting 
social needs in China 


i ! 

i i 


19.3 j 


24,4 ; 


7.7 


48.5* 


Knowledge 


F 


understand the importance of terracing 
in China's farming 


| 30 

i I 


5.5 j 

i 
■ 

i 


15.0 j 


5.3 


74.0* 


Knowledge 


F 


know why Canada - China twinning 
is important 


I 31 : 

I ; 
I 1 


4.1 


55.4* 


33.3 


V.O 

■ 


Knowledge 


F 


understand why China is "opening to 
the world" 


| 3* 


24.4 

i 


8.3 


62.r 


5.0 


Knowledge 


F 


understand the importance of the Pacific Rim 


i 34 

| 
1 


7.9 


9 4 


24.1 


58.4* 

i 


Knowledge 

I 
I 


F 


know which province borders on the Pacific 
Ocean 


i 35 


82.9* 


3.8 


4.2 


i 

1 9.0 

i 


I 

j Knowledge 


F 


know that silk originated in China 


i- - 

: 13 

j 


7.3 




47.0* 


4.3 


41.3 


i Process Skills 

i 


L 


| 

interpret comments about technology 

i 


- 14 

I 


23.9 


,, 7 


13.0 


45.3* 


j Process Skills 


L 


1 

j interpret Chinese comments about Canada 


28 


14.0 


71.0* 


2.8 


I 

! 12.1 

i 


j 

j Process Skills 


L 


1 interpret a chart to select main idea 

I 


; 29 

i 
i 


10.9 


7.1 


79.5* 


| 2.4 

i 


j 

Process Skills 


L 

I 


interpret a chart to apply its information 


i 43 

| 


! 54.6* 


15.8 

1 


j 16.7 


! 12.6 


i Process Skills 


; L 

i 


j 

j locate a Pacific Rim country on a map 


i 12 


12.7 


6.2 


i 

71.3* 


9.7 


Process Skills 


| A 


i evaluate information from different speakers 1 
1 comments 


i 15 

i 


; 54.8* 

i 


\ 27.2 


j 5.2 


j 12.7 


| Process Skills 


I A 


j evaluate statements about Chinese policy 



*correct answer 

Category Legend: F— Facts 

C —Concepts 
G— Generalizations 



L— Locating, Organizing, Interpreting Information 
A— Analyzing, Synthesizing, Evaluating 
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Multiple-Choice 
Reporting Categories 

Table 3-7 shows the total marks 
possible and the raw score 
averages for the multiple-choice 
reporting categories. 

It is important to stress that the 
averages in the various reporting 
categories cannot be directly 



compared with one another. 
Rather, the results shown in Table 
3-7 can best be used in conjunction 
with parallel tables in the 
jurisdiction and school reports. 
Variations in patterns of results 
can help indicate strengths and 
weaknesses in local educational 
programs. 



Statistics presented in this table 
are based on results achieved by 
30 763 students (those who wrote 
the regular form and those who 
wrote Form F of the test). 



Table 3-7 

Grade 6 Social Studies 

Ra>v Score Results by Multiple-Choice Reporting Category 

June 1993 





Maximum 


i 

i 
I 


Raw Score 




Possible 


Raw Score 


Standard 


Reporting Category 


Raw Score 


Average 


Deviation 


Topic A— Local Government 


18 


10.9 


3.2 


Topic B— Greece: An Ancient Civilization 


15 


10.0 


2.5 


Topic C— China: A Pacific Rim Nation 


14 


8.9 


2.8 


Knowledge: (Facts, Concepts, Generalizations) 








Topic A 


8 


5.4 


1.8 


Topic B 


9 


6.7 


1.6 


Topic C 


7 


4.6 


1.7 


Process Skills: 








Locating/Organizing/Interpreting 


13 


6.9 


2.4 


Analyzing/Synthesizing/Evaluating 

\ 


10 


6.2 


2.0 


Major Components: 




i 
I 

i 


Knowledge 


24 


16.7 


I 4.1 


Process Skills 


23 

i 
• 


13.1 


j 3.8 


j Total Multiple Choice 

■ 


I 47 

• 


29.8 


j 7.3 

I 



o 

FRir 



0 
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Results for 

Part B: Written Response 

The number and percentage 
distribution of scores, by written- 
response reporting category, are 
presented in Table 3-8. These data 
are based on the results achieved 
by all 32 630 students who wrote 
English forms of the 1993 Grade 6 
Social Studies Achievement Test. 
The average weighted score on the 
written-response portion of the test 
was 16.5, with a standard 
deviation of 4.9. 

The percentage of students 
achieving standards on "State and 
Support an Opinion/Position" was 
only 58.4% compared to 41.6% who 
failed to achieve the acceptable 



standard; 72.7% of students 
achieved standards on "Quality of 
Language and Expression" 
compared to 27.4% of students who 
did not achieve standards. See 
page 41 for the Examiner's 
Observations on the written 
response. 

Although the papers were scored 
on a one-marker system, randomly 
selects d papers were re-marked so 
that a second set of scores was 
available to confirm scoring 
consistency. Of the scores awarded 
on the second reading, 42.4% were 
identical to the scale scores 
originally awarded and 46.5% 
varied by only one point. The one- 
marker system produces results 



that are reasonably reliable for 
groups of 25 or more students. 
Achievement test scores, however, 
are less reliable for individual 
students. 

It is possible to draw conclusions 
about program strengths and 
weaknesses by comparing local 
percentage distributions of scale 
points in each reporting category 
with provincial distributions. The 
comparisons are most clearly 
understood in the context of the 
assignment students responded to 
and the scoring guides used to grade 
their work. Copies of the writing 
assignment and scoring guides are 
available from Student Evaluation 
Branch, Alberta Education. 



Table 3-8 

Grade 6 Social Studies 

Number and Percentage Distribution of Scores by Written-Response Reporting Category 

June 1993 



Score Scale Points 



(Excellent) 
4 

(Good) 
3 

(Acceptable) 
2 

(Limited) 
1 

(Poor) 

INS 
(Insufficient) 



State and Support an Opinion/Position 



Number 

923 
4304 
13847 
11060 
2355 
121 



Percent 



2B 



42A 



34.0 



12. 



04 



Quality of Language and Expression 



Number 



970 



4764 



17976 



7500 



1299 



121 



Percent 
3.0 
14.6 
55.1 
23.0 
40 
04 



Examiner's Observations 

Achievement in Grade 6 Social 
Studies is slightly better than 
1989, However, not enough 
students achieved either the 
acceptable standard or the 
standard of excellence on the total 
test. 



Multiple Choice 

Students found the process skills 
questions on the multiple-choice 
section of the test very challenging. 
Sample questions from the test and 
accompanying discussion are 
provided to highlight the strengths 
and weaknesses of students who 



achieved the acceptable standard 
and the standard of excellence. For 
each sample question, the correct 
answer is marked with an asterisk. 
The percentage of students 
choosing each alternative is also 
provided. 
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Use the following information to answer question 37. 



Acceptable Standard 



When the people of Big Lake discussed the need for a swimming 
pool, some supported the idea while others were against it. The 
town council voted in favour of building the swimming pool. 




37. If the town council does not change its decision, Mrs. Jones has a responsibility to 

6.6 A. learn to swim 

10.5 B. refuse to pay her taxes 
8.2 C. refuse to use the i>ew pool 

74.6 *D. accept the council's decision ever) if she is upset 




"It is the opinion of our 
f government that foreign visitors, goods. N 
and money influence our you lh. Young 
people must learn the benefits of life 
tn China and of living in 
a Chinese culture.' 




"Trade with western 
countries is very importnnl to the 
" Chinese economy. Wc liavc many products ^ 
( that countries like Canada and the United States^ 
L need and want. These countries arc willing to y 
pay a great deal of money for them, 
which can. only benefit the 
Chinese economy.* 




Sin 

Minister of Culture 



Han 

Businessman 



15. Sin would most likely agree with which statement? 

54.8 *A. The government should strongly promote Chinese values among its young people 
27.2 B. The government should encourage bringing cultural groups into China. 
5.2 C. Chinese people should be allower] to buy whatever they like. 
12.7 D. China should import more produ' Is to change young people 



For question 37, students needed 
to have a basic understanding of 
rights and responsibilities. 
Students achieving the acceptable 
standard have a good 
understanding of the necessary 
balance between rights and 
responsibilities in local 
government issues. 

For question 15, students were 
required to analyze information 
and draw an inference. Results 
indicated that students who 
achieved the acceptable standard 
but not the standard of excellence 
had difficulty making inferences. 
However, students achieving the 
standard of excellence were clearly 
able to do so. 

The strengths of students who 
achieved the acceptable standard 
include an ability to: 

• apply their knowledge of basic 
concepts such as human needs, 
class structure, rights, 
responsibilities, lobby groups, 
and sharing 

• synthesize information to draw 
simple conclusions 

• compare and contrast ways 
different societies meet needs. 

Many students, however, did not 
do as well as expected in: 

• recalling some basic knowledge 
as outlined in the course of 
study, such as knowledge about 
Pacific Rim and government 

• analyzing and synthesizing 
information to make inferences 
and determine the main idea 

• comparing information, charts, 
anC graphs 

• distinguishing between fact and 
opinion 

• comparing maps of different 
scales and computing distances. 
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11. The picture below that shows what the Ancient Greeks would have used to satisfy a 
physical need 

7.9 B. 

\ ccccccc — 00 
i occ-ccco-ao 

Abacus beads 




14.3 



> //a 



21.2 D. 



V I 



Scroll 



Musical instrument 



Use the following information to answer question 41. 



The people of Brownsville are concerned about the increase in bicycling 
accidents over the last two years, as shown in the following chart and graph: 

Accidents Involving Bicycles 



Type of Accidents 

Automobile and bicycle accidents 
Pedestrian and bicycle accidents 
Individual bicycle accidents 
Property damaged by bicycles 



Number of Accidents 



1991 

13 
29 
10 
9 



1992 

17 
36 
31 
20 



Accidents Involving Bicycles 



Number of 
Accidents 



35 

30- 
25- 
20- 
15 - 
10 



1 88 ij 



199) 1992 
Automobile 



1991 1992 
Pedestrian 



1991 1992 
Individual 



1L 

1991 1992 
Property 



Type of Accidents 



Note to the reader; Pedestrian bars were intentionally reversed for another test question that 
probed the display of data 

41. A group who could best use the statistics from the chart would be 

25.3 A. hospital managers requesting increased funding 

36.5 *B. citizens who support a bylaw to restrict bicyclists 

17.0 C. citizens who oppose a bylaw to restrict bicyclists 

20.9 D. police officers when they charge bicycle offenders 



Standard of Excellence 

For question 11, students had to 
have a basic understanding of 
human needs. A high percentage of 
students achieving the standard of 
excellence were able to identify a 
physical need. However, only 
slightly more than half of students 
achieving the acceptable standard 
were able to identify how the 
ancient Greeks satisfied a physical 
need. 

For question 41, students needed 
to make inferences by analyzing 
and comprnng information 
contained in a chart and a graph. 
Many students who achieved the 
acceptable sto ndard and some 
students who achieved the 
standard of excellence had 
problems selecting the correct 
inference. Too many students in 
both groups were drawn to the first 
alternative. 



Students who achieved the 
standard of excellence were able to: 

• clearly understand concepts as 
outlined in the course of study 

• select main ideas by reading 
information 

• analyze and synthesize 
information to make simple 
inferences and draw conclusions 

• identify points of view 

• compute distances on maps. 

However, some of those students 
did not do as well as expected in: 

• distinguishing between fact and 
opinion 

• making an inference by 
comparing information from a 
chart and a f • ■ aph (see question 
41) 

• comparing maps of different 
scales. 
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Written Response 

Although the quality renting in 
social studies is slightly better in 
1993 than it was in 1989, not 
enough students achieved the 
acceptable standard. Teachers 
were disappointed with many of 
the papers that they marked. The 
markers' specific observations 
include the following: 

• too few students handled the 
composition adequately 

• many students had difficulty 
organizing ideas 

• writing was general rather than 
specific and some students used 
unrelated personal examples 

• some students showed a lack of 
understanding of government 
and some confused the levels of 
government 

• many students had difficulty 
communicating ideas in support 
of their position 

• a small number of excellent 
papers were written 

A majority of teachers indicated 
that the writing task may have 
been too general and, rather than 
delimiting the topic, many 
students responded with broad 
statements that did not 
communicate clearly or effectively. 
These concerns will be addressed 
in future designs for written- 
response questions. 

Issues 

During the 1993 Grade 6 Social 
Studies marking session, teachers 
were asked to comment on the 
writing assignment. The markers 
made a number of suggestions 
regarding students' writing in 
support of a position. For example, 
they indicated that taking a 
position on an issue should follow 
the generation of ideas and 
considerntion of alternatives. 



Qualified positions can be 
supported as well as "for" and 
"against" positions. Students in 
Grade 6 classrooms can best 
address issues when they are 
specific and in familiar contexts. 

Performance-Based 

Assessment 

General Description 

A total of 79 Grade 6 teachers and 
1 809 students from a sample of 
schools throughout the province 
were involved in an assessment of 
process skills. Student and teacher 
survey instruments were 
constructed to help assess the 
extent to which the identified 
processes have been acquired by 
students. The results are presented 
in Tables 3-9 (students) and 3-10 
(teachers).. 

Students were asked to complete a 
self-assessment of 11 process skills 
areas. Teachers used three 
descriptors to rate each of their 
students on the same 11 processes 
areas. 

This assessment was intended to 
help describe the relationship 
between student results on the 
1993 Grade 6 Achievement Test 
and student acquisition of process 
skills. 

Examiner's Observations 

The instruments used by the 
teachers and the students provided 
a limited amount of information 
regarding the identified process 
skills. However, many of the Grade 
6 teachers who participated in the 
study were pleased with the 
initiative and enthusiastic about 
the general trend to assess a 
broader range of skills. Some 
teachers indicated that the study 
should have been based on the 
skills as defined in the Program of 
Studies. 



Results show that students rate 
themselves highest in the 
processes of listening to others, 
persistence, questioning and 
problem posing, and 
inquisitiveness. Teachers rated 
students highest in the processes of 
questioning and problem posing, 
precision of language and thought, 
metacognition, and inquisitiveness. 
The ability to deal with several 
sources at one time and to consider 
alternative^noihts of view 
(flexibility in thinking) was the 
area rated lowest by both students 
and teachers. The results from this 
assessment will be used as a basis 
to design future performance 
assessments that reflect broad 
learner expectations in the social 
studies program. 

Relationship between 
Performance-Based 
Assessment and the 
Achievement Test 

Correlations between teacher 
ratings of students' process skills 
and the achievement test scores 
were calculated. Generally, there is 
a positive relationship between a 
student's reported attainment of 
the identified process skills and 
his/her performance on the 
achievement test. For example, 
teacher ratings of student 
metacognitive skills correlated 0.51 
with student achievement test 
scores. The relationship between 
questioning and problem posing 
skills and achievement test scores 
was correlated at 0.44, 
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Table 3 9 

Grade 6 Social Studies 

Student Survey on Performance-Based Assessment 
Percentage Distribution of Student Responses 

June 1993 

Descriptors For Rating Scale 

Often: 1 consistently demonstrate this attribute 

Sometimes: I usually demonstrate this attribute, but not with consistency 
Not Yet: 1 rarely, if ever, demonstrate this attribute 



Attributes 



1 . Continuing to work when the solution to a problem is not 
easily found 

a. I stay on task but may take a break 

b . I seek other sources of information when I need help 

c. I do not need to be reminded to complete tasks/projects 

2. Thinking before acting 

a. I listen and understand directions before starting 

b. I listen to what other students say and do not ask unnecessary 
or the same questions 

c. I ask questions to make the assignment clear to me 

d. I study the problems and develop a plan to solve them 

e. I think before answering 

3. Listens with understanding and feeling 

a. I repeat other students' ideas in my own words 

b. I use body language to respond 

c. I ask questions that are on the topic 

d. I answer with acceptable actions or words 

4. Awareness of our own thinking 

a. When faced with a problem, I can explain what the problem is 

b . I can explain the parts of the problem that !■ already know 

c. I can identify the parts of the problem that I need to know 

d. I develop and carry out plans to solve problems 

e. I change my plan when it needs to be changed 

f. I think about what I did to solve the problem 



I 5. 



Language is clear, descriptive and complete 

a. I use specific words 

b . I choose the best possible words to describe my speech 

c. It is important for me to be understood 

d. When needed, I clearly express my ideas in detail 

The ability tc deal with several sources at one time and to 
consider alternative points of view 

a. I am willing to change my mind when given good reasons 

b. I offer and accept more than one alternative to problems 

c. I am willing to accept there may be more than one answer to 
a problem 

d. I am willing to accept other people's solutions to problems 



Rating Scale 


i 






No 


Often | 


Sometimes 


Not Yet 


Response 


29.5 


64.8 


5.0 


0.7 


39.5 


55.2 


4.8 


0.5 


33.4 


53.9 


11.9 


0.8 


53.9 


42.7 


2.7 


0 7 


48.9 


41.2 


9.1 


0.8 


53.5 


40.3 


5.6 


0.6 


32.9 


57.0 


9.4 


0.7 


63.2 


33.3 


2.6 


0.9 


19.5 


61.1 


18.7 


0.7 


25.3 


51.4 


22.6 


0.6 


61.8 


33.6 


3.8 


0.8 


48.5 


45.9 


4.5 


1.0 


33.7 


59.8 


5.7 


0.8 


62.8 


33.8 


2.4 


0.9 


| 45.9 


48.8 


I 4.2 


1.1 


34.3 


57.3 


7.3 


1.1 


! 54.1 


i 39.5 


5.3 


1.0 


j 36.6 


49.4 


13.2 


0.8 


j 31.3 


62.6 


5.2 


0.8 


38.7 


54.1 


6.4 


0.8 


67.7 


27.9 


2.9 


1.5 


40.8 


51.0 

I 


6.9 

i 


1.3 


! 58.3 


37.8 


| 3.1 


0.8 


] 42.4 


50.5 


I 5.8 


1.3 


63. 0 


j 32.7 


I 3.1 

I 


1.2 

i 


j 58.0 

i 
1 


37.3 


| 3.0 


1.8 



Continued 
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Table 3-9 (continued) 





Rating 


Scale 


Attributes 


Often 


Sometimes 


Not Yet 


No 
Response 


7. Has the interest and ability to find problems to solve 

a. 1 am not afraid to ask questions 

b. My questions are serious ones that 1 really want answered 

c. 1 ask questions until 1 am pleased with the answer 

d. Some of my questions are much harder than others 


48.0 
43.0 
33.2 
34.2 


42.9 
51.0 
49.8 
52.9 


8.3 
4.9 
15.8 
11.8 


0.8 
1.0 
1.2 
1.0 


8. All aspects of work are reviewed and checked 

a. 1 reread instructions and make sure everything has been done 

b . I want all my work to be my best 

c. 1 try to correct errors before I finish a task 


38.0 
63.6 
44.5 


54.5 
33.S 
48.6 


6.5 
1.9 
5, 


1.0 
0.9 
1.2 


9. The ability to understand one experience and apply it to 
new situations 

a. In a new situation, 1 figure out what I already know 

b . ! remember and use important previous experiences 

c. 1 use ideas from the past to help me solve new problems 


42.2 
45.6 
49.4 


52.8 
48.1 
44.2 


4.0 
4.9 
5.3 


1.0 
1.3 
1.1 


10. Creativity 

a. I do work and activities because I want to 

b . I am willing to try things even when I am unsure of the result 

c. I want others to tell me how I am doing 


49.2 
51.5 
48.6 


44.1 
43.5 
40.9 


5.7 
4.0 
9.2 


1.0 
1.0 
1.3 


11. Feel good about our thinking ability 

a. I enjoy finding problems to solve 

b . I believe I can solve most problems 

c. I am curious and interested in most things 


23.8 
33.6 
62.1 


55.6 
57.4 
33.3 


19.5 
8.9 
3.6 


1.1 
1.1 
1.0 
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Table 3-10 

Grade 6 Social Studies 

Teacher Survey on Performance-Based Assessment 
Percentage Distribution of Teacher Responses 

June 1993 



Descriptors For Rating Scale 

Often: Student consistently demonstrates this attribute 

Sometimes: Student usually demonstrates this attribute, but not with consistency 
Not Yet: Student rarely, if ever, demonstrates this attribute 



Attributes 



1 . Persistence: 

Continuing to work when the solution to a problem is not easily found 

a. stays on task but may take a break 

b . seeks other sources of data 

c. works independently to complete tasks/projects 

2. Decreasing Impulsivity: 

Thinking before acting 

a. listens and understands directions before starting 

b . listens to the responses of others: therefore reduces the number 
of unnecessary or repetitive questions 

c. asks questions to clarify the task or direction 

d. analyzes the problem and develops a plan 

e. thinks before answering 

3. Listening To Others: 

Listens with understanding and empathy 

a. paraphrases others' responses 

b . uses body language to respond 

c. asks questions related to the topic 

d. responds by acceptable actions or words 

4. Metacognition: 

Awareness of our own thinking 

a. verbalizes an understanding of the problem 

b . describes what is known to solve the problem 

c. identifies what else needs to be known 

d. develops and carries out a plan of action 

e. recognizes when the plan may need modification 

f. reflects/evaluates strategies used to solve the problem 

5. Precision to Language and Thought: 

Language is clear, descriptive and complete 

a. uses specific words (nouns, etc.) 

b . descriptors are used to enhance speech 

c. strives for coherence and conciseness 

d. elaborates and clarifies ideas 



Often 



46.0 
34.4 
50.0 



49.3 

48.5 
43.8 
39.8 
49.8 



24.0 
33.9 
46.4 
49.8 



39.0 
40.6 
35.8 
41.1 
33.0 
27.2 



Rating Scale 



Sometimes 



43.2 
49.0 
37.2 



43.9 

41.5 
43.9 
44.8 
41.1 



53.2 
53.7 
42.7 
42.7 



48.6 
47.2 
47.9 
45.0 
45.0 
45.3 



Not Yet 



10.4 
16.1 
12.3 



6.3 

9.2 
11.8 
14.7 

8.2 



22.3 
11.6 
10.4 
6.9 



11.8 
11.6 
15.5 
13.3 
21.4 
26.6 



No 

Response 



0.5 
0.5 
0.5 



0.5 

0.7 
0.5 
0.8 
1.0 



0.5 
0.8 
0.5 
0.6 



0.6 
0.6 
0.9 
0.6 
0.6 
0.9 



39.6 I 


51.1 


8.5 


0.6 


32.1 1 


50.4 


16.8 


0.7 


37.5 i 


45.2 


16.6 


0.8 


31.7 I 


47.4 


20.0 


0.9 
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Continued 



Table 3-20 (continued) 



Attributes 



Rating Scale 



Often 



6. Flexibility In Thinking: 

The ability to dealwith several sources at one time and to consider 
alternative points of view. 

a. is willing to change his/her mind when presented with reasons 
to do so 

b. offers/accepts more than one alternative to a problem 

c. is willing to accept that there may be more than one answer 

d. is able to compromise (give up ownership) 

7. Questioning And Problem Solving: 

Has the inclination and ability to find problems to solve 

a. student is not hesitant to ask questions 

b . questions result from critical observations 

c. questions openly, is willing to probe 

d. demonstrates different levels of questioning 

8. Checking For Accuracy And Precision: 

All aspects of work are reviewed and checked 

a. rereads instructions and makes sure nothing has been 
omitted 

b . takes pride in the accomplishment 

c. strives for quality through editing and proofing 

9. Drawing On Past Knowledge and Applying it To 
New Situations: 

The ability to abstract meaning from one experience and apply it 
to new situations 

a. determines what is known about a situation 

b . takes meaning from relevant past experiences 

c. appropriately applies strategies used in the past in a new 
situation 



10. 



11. 



Ingenuity, Originality And Insightfulness: 

Creativity 

a. student is self-motivated 

b . takes risks and tests limits 
seeks and accepts feedback 



c. 



Wonderment, Inquisitiveness, Curiosity And The 
Enjoyment Of Problem Solving: 

A sense of efficacy as a thinker 

a. student derives pleasure from being a problem seeker 

b. student has confidence in his/her ability to solve problems 

c. has a curiosity about and an affinity with the world and the 
environment 



53.1 
52.3 
58.8 
53.3 



49.2 
35.1 
38.0 
29.2 



35.9 
50.5 
34.7 



42.6 
43.3 

38.6 



47.1 
32.4 



Sometimes! Not Yet 



No 
Response 



42.2 
39.2 
35.5 
39.1 



38.4 
45.7 
39.4 
44.7 



49.0 
39.9 
42.7 



50.1 
47.4 

48.8 



38.3 
46.5 



3.8 
7.4 
4.5 
6.4 



11.6 
18.3 
21.4 
24.9 



14.2 
8.9 
21.6 



5.4 
7.7 

10.7 



13.9 
20.4 



41.4 


43.1 


i 14.6 

I 


0.9 


34.6 


46.1 


j 

18.6 


0.8 


37.2 


43.1 


; 18.9 


0.8 


45.4 


43.4 


; 10.1 


1.1 



0.9 
1.1 
1.1 
1.2 



0.8 
0.9 
1.1 
1.3 



1.0 
0.8 
1.1 



1.8 
1.7 

1.9 



0.7 
0.8 
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Contexts for Learning 
General Description 

In June 1993, a sample of Grade 6 
Social Studies teachers and 
students from across the province 
completed separate questionnaires 
designed to identify and examine 
relationships among various 
contexts for learning and their 
effect on student achievement. 

Teachers reported on the variety of 
instructional strategies, writing 
activities, assessment tools, 
classroom resources, and other 
activities they used in their social 
studies classes. The results are 
presented in Table 3-11 (teachers). 

Students gave their opinions on a 
variety of topics and on various 
learning activities related to social 
studies. The results are presented 
in Table 3-12 (students). 



Teachers indicate that they use a 
wide variety of social studies 
related activities in their Grade 6 
classes. These activities includes 
co-operative learning, 
brainstorming, note taking, and 
class discussion. 

Student responses to the 
questionnaire indicate that 
students have a marked tolerance 
and openness to people different 
from themselves. A large majority 
of students indicated that they are 
accepting of differing opinions and 
of the importance of being 
responsible citizens; 84% of the 
teachers surveyed indicated that 
most of their students show respect 
for other's opinions or viewpoints. 

Students responses were highest in 
the area of cooperation. For 
example, to the statement "It is 
important for me to do my share in 



group projects", 95.7% of the 
students "agreed." In a related 
area, 92.0% of the teachers 
surveyed indicated that most of 
their students demonstrated 
courteous behaviour when working 
with others. 

Relationship between 
Learning Environment 
and the 

Achievement Test 

No relationship or pattern was 
established by students achieving 
or not achieving the acceptable 
standard and their responses to 
the student questionnaire. It is 
also interesting to note there was 
no discernable pattern between 
achievement and instructional 
strategies. 



Table 3 11 

Grade 6 Social Studies 

Teacher Questionnaire — Contexts for Learning 
Percentage Distribution of Responses 

June 1993 



: 1. 









Less than 


Once 
a 


Several 








Statement 




once a 


times a 




No 






Never 


week 


week 


week 


Every day 


Response 


I use the following instructional strategies 














with my Social Studies students. 












i 


a. 


Full class discussion 




— 


16.0 


52.0 


32.0 


1.0 


b. 


Reading from a textbook 




32.0 


20.0 


48.0 


1.0 




c. 


Worksheets 


12.0 


32.0 


24.0 


i 28.0 




4.0 


d. 


Cooperative learning 




20.0 


20.0 


! 52.0 


! 8.0 


_ ! 


e. 


Library research 


| 4.0 


68.0 


16.0 


| 12.0 






X 

1 . 


Viewing films or videos 




80.0 


12.0 


! 8.0 




! — i 


g. 


Oral presentations 




64.0 


16.0 


; 20.0 






h. 


Project work 




52.0 


24.0 


i 16.0 


] 4.0 


! 4.0 


i. 


Note taking 




| 36.0 


36.0 


j 28.0 


j - 




J- 


Brainstorming 


i 

i 


| 20.0 


| 28.0 


! 44.0 


| 8.0 




k. 


Webbing/mind mapping 




38.0 


; 12.0 


! 20.0 


I - 

I 




1. 


Learning or listening centres 


' 56.0 


j 28.0 




\ 4.0 


! 4.0 


! 8.0 


m 


Independent work 




8.0 


48.0 


36 0 


8.0 




n. 


Problem-solving 




20.0 


i 32.0 


; 40.0 


; 4.0 


i 4.0 | 


0. 


Teacher-directed inquiry 




1 16.0 


i 32.0 


| 44.0 


j 4.0 


! 4.0 


P- 


Student-directed inquiry 




; 40.0 


; 24.0 


' 24.0 


i 4.0 


8.0 ! 


q- 


Peer discussion 




20.0 


! 20.0 


! 44.0 


1 8.0 


1 8.0 
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Table 3-11 (continued) 



I 

! 

I 

j 


\ 

Statement ! 

! 

i. 


Never 


Less than 
once a 
week 


Once 
a 

week 


1 

Several 
times a 
week 


Every day 


No 
Response 


i 

! 2 


i i 
My class doGs thG following kinds of writing in 


i 




1 








! 
i 
! 
















i 
1 


a. Paragraphs 




28.0 


40.0 


24.0 


z 


8.0 


j 


.b. Reports 


— 


68.0 


12.0 


12.0 




8.0 




c. Series 


24.0 


60.0 


4.0 


- 


4.0 


8.0 


| 


d. Writing about a personal experience 


24.0 


64.0 


4.0 




4.0 


4.0 




e. Copying notes from the board 


12.0 


52.0 


28.0 


8.0 






i 


f. Notes from class/group discussion 




40.0 


24.0 


32.0 


— 


4.0 


! 


g. Letters 


32.0 


52.0 


8.0 






8.0 


> 

i 


h. Summarizing information from a variety of 


4.0 


28.0 


32.0 


36.0 


— 




i 


sources 
















i. Interviews 


32.0 


56.0 


4.0 




- 


8.0 


| 3. 


i integrate the racial Studies program with other 


4.0 


36.0 


20.0 


24.0 


12.0 


4.0. 


i 


curriculum areas. 












i 




1 encourage students to make decisions and 




16.0 


16.0 


32.0 


36.0 




s 
1 


consider the consequences. 




I 










1 5. 

L 


1 encourage students to take action on an issue. 




I 44.0 

i 
I 


| 20.0 


12.0 


24.0 





S^tement 



Yes 



Most of my students demonstrate courteous 
behaviour when working with others. 

Most of my students can accept the role of 
either leader or follower in a group. 

Most of my students show respect for someone 
else's opinions or viewpoints. 

The following activities are part of my grade 6 
Social Studies program. 

a. Field trips 

b . Simulation, role-play, or cooperative 
games 

c. Listening to guest speakers 

d. Interviewing parents and/or members of the 
community 

e. Special celebrations 

f. Making maps, charts, timelines, 
and/or graphs 

g. Writing a letter 



92.0 



76.0 



84.0 



No 



16.0 



8.0 



72.0 
84.0 

96.0 
64.0 

64.0 
100.0 

72.0 



28.0 
16.0 

4.0 
36.0 

32.0 



28.0 



No Response 



8.0 



8.0 



8.0 



4.0 
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Table 3-11 (continued) 



Statement 

i 


I 

Yes i 

i 


No 


No Response 




1 10. I use the following types of assessments with I 


i 

* ! 

I 


j 








my Social Studies students. j 

i 


i 
i 










a. Multiple choice tests i 


1 

8t.O 


12.0 j 








h \/r»f**ak*ii iIqi-v matrh 
U. VUv^aUUIdiy llldlOM 


84.0 


16.0 ; 








» c Shnrt an^wpr I 


96.0 


4.0 \ 








; d. True-false | 


84.0 


12.0 | 


4.0 




; e. Extended response 


96.0 


4.0 








f . Interpretations of charts, diagrams, 


100.0 


0.0 








! pictures, graphs, and maps 












g. ureaxion ot a oiagram t picture, murai, 


oo.u 


19 0 

I C-.\J \ 








1 collage, or model to show understanding 












! h. Oral presentation 

1 


96.0 


4.0 




I 

I 


! 


Students 


Teachers 






Statement 


Use 


Use 


twin 


neuner 


1 1 . The following items were readily available 










! in my classroom for my students and/or for 










my use in planning and instructing. 










a. Nelson Intermediate Atlas 


24.0 


12.0 


52.0 


12.0 


b. Dictionary 


36.0 




64.0 




c. Encyclopedias 


16.0 


16.0 


36.0 


32.0 


d. Globe 


28.0 


8.0 


56.0 


8.0 


e. Newspapers 


28.0 


4.0 


40.0 


28.0 


f . Magazines 


24.0 


16.0 


28.0 


32.0 


g. Travel Brochures 


20.0 


16.0 


20.0 


44.0 


h. Films, videos 


32.0 


16.0 


24.0 


28.0 


| i. Maps 


28.0 




64.0 


8.0 


j j. Computer 


28.0 


j 4:0 


36.0 


32.0 ; 


| k. Topic A: Basic Resources 








! 


i. Politics and You (student text) 


' 56.0 


I - 


! 24.0 


: 20.0 j 


; ii. Politics and You (teacher's guide) 


! 16.0 


] 56.0 


< 


! 28.0 


| iii. The Structure of Government (kit) 


20.0 


I 44.0 


! 8.0 


i 28.0 


'* iv. The Winds of Change: Indian 


J 20.0 


| 20.0 


| 16.0 


! 44.0 


j Government (student texts) 


I 




I 




I v. Working for Canadians: A Study of 


; 32.0 


, 12.0 


24.0 


32.0 


Local, Provincial , and Federal 


! 


I 


i 




Government (student texts) 


i 


i 




! 
• 


I. Topic B: Basic Resources 






i 




! i. The Greeks; People of the Past 


I 52.0 


i 8.0 


! 32.0 


i 8.0 



series (student texts) 

m. Topic C: Basic Resources : 

i. Life in Changing China (student j 56.0 
texts) | 

ii. Life in Changing China (teacher's ! 28.0 
manual) j 

n. Social Studies Teacher Resource Manual, 20.0 
Grades 4-6 

0. Focus on Research: A Guide to Developing ! 8.0 

Students' Research Skills I 

p . Teaching Thinking: Enhancing Learning | 4.0 



4.0 
60.0 
68.0 
36.0 
20.0 



36.0 
8.0 
4.0 
4.0 
4.0 



4.0 
4.0 
8.0 
52.0 
72.0 
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Table 312 

Grade 6 Social Studies 

Student Questionnaire — Contexts for Learning 
Percentage Distribution of Responses 

June 1993 





Statement 


I 
j 

! Disagree 


i 
i 

Undecided j 

i 


Agree 1 

\ 


No 
Response 


1. 


Participation is an important part of Socia! Studies. 




■ 1 

11.3 


i 

87.6 j 




: 2. 

i 


It is important for me to do my share in group 
projects. 


! 1.4 

I 


2.8 


95.7 ! 

I 


0.1 


3. 


Social Studies has been interesting this year. 


| 12.3 


34.4 


53.3 




4. 


Social Studies is a difficult subject. 


! 34.3 


39.5 


26.0 


0.1 


1 5. 


Social Studies is useful for solving problems and 


j 22.7 


36.5 


40.5 


0.3 



I 



dealing with situations in everyday life i I | 

j j 

6. Violence should not be used to solve disputes 4.5 j 10.7 \ 84.7 
between countries. j ! j 

7. Immigrants to Canada offer points of view that make ; 9.2 | 41.2 j 49.2 
Canada a better place. 

8. Other people have a right to an opinion that is 1 .0 i 3.6 ! 95.2 
different from mine. 

] 9. A person who moves to Canada from another culture 82.5 \ 12.1 i 5.1 
| should give up his/her own culture. 

; 10. I think that it is important to know about other 2.3 j 12.0 j 85.0 

countries and what is going on in the world. j ; i 

! ! ! ! 

111. If people work together, they can help solve some I 1.7 ! 8.5 | 89.8 

| of the world's problems. • j 

I 12. When people wish to change a law, they should j 6.3 j 40.6 ! 52.6 

; follow the established democratic practices, j j ! 

I no matter how long it takes. j j 

13. There is no point in trying to think of solutions to local j 73.6 , 20.1 6.1 
j problems because they are too complicated. | I 

i ! i 

■ 14. The responsibility for students doing well in school j 5 3 i 16.O I 77.3 
j lies mainly with the student him/herself. j j 

i 15. Citizens of Canada are responsible for seeing that 3.6 9.9 86.2 

Canada's environment is kept clean. 

.16. It is important that I become a responsible citizen, j 1- 7 | 90 ; 89 - 3 

17. People should not have to look after each other. ; 48.3 30.9 20.1 

They should look after themselves. 



0.4 
0.3 

0.3 
0.7 



0,4 

0.3 

0.4 

0.3 

0.1 
0.7 
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Table 3-12 (continued) 



Statement 


Disagree 


Undecided 


Agree 


No 
Response 


18. 


1 think that it is important to know about 
Canada's past and present. 


5.4 


18.5 


75.8 


0.4 i 

I 


19. 


People in Canada can meet their needs in 
different ways. 


1.5 


11.8 


86.1 


0.6 


20. 


The way that people meet their needs in China is 
just as good as the way we meet our needs in 
Canada. 


15.7 

: 


33.3 

I 
i 

1 


50.8 

! 

! 


0.1 
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Section 4 



Grade 9 
Science 



In 1993 a number of assessments 
were carried out in Grade 9 
Science. The achievement test was 
administered to students province- 
wide. Performance assessment 
tests were administered to a 
sample of Grade 9 Science 
students. As well, a sample of 
Grade 9 Science teachers and a 
sample of Grade 9 Science students 
participated in a study to examine 
the relationship among various 
contexts for learning and 
achievement. The results *rom all 
these assessments follow 



Achievement Test 
General Description 

The Grade 9 Science Achievement 
Test was divided into two parts. 
Part A had 60 multiple-choice 
questions, each with four 
alternatives. Part B had 5 
numerical-response questions, 
which required students to 
determine or calculate an answer 
and then record it on the answer 
sheet. 

The statistics for the total te*t and 
for the various components are 
based on the results achieved by 
28 613 students: 25 635 wrote the 
regular form and 2 978 
participated in the achievement- 
over-time study, as shown in Table 
4-2. This section of the report 
provides answers to the following 
questions: 



° How many Grade 9 students 
wrote each form of the test or 
were absent or exempted? 

• What percentage of students who 
wrote the Grade 9 Science test 
achieved the acceptable 
standard? 

•What percentage of students who 
wrote the Grade 9 Science test 
achieved the standard of 
excellence? 

• What did Grade 9 students know 
and what could they do in 
Science? 

•What parts of the Science 
curriculum caused Grade 9 
students difficulty? 



Summary of Results 

Results show that 77.6% of 
students who wrote the test 
achieved the acceptable standard 
and 17.1% achieved the standard 
of excellence on the total test. The 
number of students achieving the 
acceptable standard was lower 
than expected and the number 
achieving the standard of 
excellence was higher than 
expected. 



Content of the Test 

The Grade 9 Science Achievement 
Test was designed to reflect the 
Grade 9 Science curriculum 
standards. The scope of the test 
was limited to student learner 



expectations that could be 
efficiently measured on a paper 
and pencil test. 

The test consisted of 65 questions 
in six topics: Diversity of Living 
Things, Fluids and Pressure, Heat 
Energy: Transfer and 
Conservation, Electromagnetic 
Systems, Chemical Properties and 
Changes, and Environmental 
Quality. The questions were 
designed to measure achievement 
in two domains: Concepts and 
Skills. 
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Test Blueprint 

Table 4-1 presents the blueprint 
used to develop the Grade 9 
Science Achievement Test. 
Classification of each question by 
component and reporting category 
is indicated in the table. 



Table 4 1 
Grade 9 Science 
Achievement Test Blueprint 

June 1993 



Assessment Component 



Diversity of Living Things 

! 

j Fluids and Pressure 

i Heat Energy: Transfer and 
j Conservation 

j Electromagnetic Systems 

| Chemical Properties and Changes 

I Environmental Quality 

\ 

i 

i Total Number of Questions and 
- Test Emphasis 

*NR— numerical-response question 



Distribution of Questions by 
Reporting Category 



Concepts 



1,5,10,11,12,14,56 
7, 8, 9, 34, 48, 49, 51,58 
15,16,17 

19, 39, 40,59, 60, NR1* 
20, 21,23,24, 33, 38, 57 
13, 28, 29, 30,53 



36 (55%) 



Skills 



2, 3,4, 6,52 
32, 35, 36, NR2 
18, 45, 46, 50, NR3 

41,42, 43, 44,47 
22, 37, NR4, NR5 
25, 26, 27, 31,54,55 



29 (45%) 



I 
j 

! Total Number of 
-\ Questions and 
: Test Emphasis 



12(18%) 
12(18%) 
8(13%) 

11 (17%) 
11 (17%) 
11 (17%) 



65(100%) 
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Student Participation 

In June 1993, principals reported a 
total population of 34 362 students 
in Grade 9. Table 4-2 presents the 
number and percentage 
distribution of students who wrote 



the Grade 9 Science Achievement 
Test, who were absent, or who 
were exempted. In total, 87.8% of 
the students reported to be in 
Grade 9 in June 1993, wrote the 
Grade 9 Science Achievement Test. 



Results for students in French 
Immersion or Francophone 
programs were reported separately 
to participating schools and 
jurisdictions. 



Table 4-2 
Grade 9 Science 
Student Participation 

June 1993 



Category 


■ 

Number j 
or students ■ 


I 

Percentage i 
ot otuoents 


Total Number of Students In Regular Programs* 


28 613 


83.3 


Students Who Wrote the Regular Form 


25 635 


74.6 


Students Who Wrote the Achievement- Over-Time Fcrms: 


2 978 


8.7 


Form T (Green)— Same as the Regular Form 


1 018 


3.0 


| Form S (Pink) 


981 


2.9 


Form R (Blue) 


979 


2.8 


Other Students Who Wrote: 


1 527 


4.5 


! 

j Francophone— Taught in French, Wrote in French 


123 


0.4 


j Francophone— Taught in French, Wrote in English 


0 


0.0 


I French Immersion— Taught in French, Wrote in French 


1 403 


4.1 


j French Immersion— Taught in French, Wrote in English 


! 1 


0.0 


Students Absent 


! 2 041 

i 


5.9 


Students Exempted from Writing 


| 2 181 


6.3 


Categories of Exemption: 


j 




1. Special Needs Students 


I 

, 1 171 


3.4 


2. Subject Was Not Taught This Term 


427 


1.2 


3. English as a Second Language Students 


! 298 


0.9 


4. Language of Instruction Was Not English 


j 1 


0.0 


5. Other (as approved by the Superintendent) 


! 284 


0.8 


Total Principals' Reported Population 






j Test Day, June 1 993: 


j 34 362 


100.00 


| Grade 9 Enrollment: September 30, 1992 


] 35 182 


I 



i i . I 

'Provincial results are based on test scores achieved by students who were instructed in English and who wrote the English form of the 
achievement test, and those students who were instructed in a language other than English or French and who wrote the English form of 
the test. They are indicated by the shaded area of the table. 
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Results in Relation to 
Standards 

Achievement test results are 
reported in relation to assessment 
and achievement standards. The 
Provincial Assessment 
Standard is the lowest score on a 
test that a student must achieve 
for his/her performance to be 
judged "acceptable" and/or 
"excellent" in relation to curricular 
expectations. The Provincial 
Achievement Standard refers to 
the percentage of students writing 
the test who are expected to 
achieve the Provincial Assessment 
Standard. 

Figures 4-1 and 4-2 and Table 4-3 
show the percentage of students 
achieving provincial assessment 
standards in relation to the 
provincial achievement standards 
on the total test and on 
components of the test. These data 
are based on the results of the 
28 613 students in regular 
programs who wrote the test. 

The percentage of students 
achieving the acceptable standard 
was lower than expected and the 
percentage of students achieving 
the standard of excellence was 
higher than expected. 

When interpreting these findings, 
it is important to remember that 
they are based on the test scores of 
only a portion (83.3%) of the 
students reported to have been in 
Grade 9 in June 1993, i.e., those 
students in regular programs who 
wrote the achievement test. These 
figures tell us nothing about the 
levels of achievement of students 
in Francophone and French 
Immersion programs, of students 
who were absent on the day of 
writing, or of those students who 
were exempted from writing. 



Figure 4-1 
Grade 9 Science 

Percentage of Students Achieving Acceptable Standard on the 
Total Test and on Components of the Test 

June 1993 



100- 



80- 
60- 

40- 
20- 






Concepts 



Skills 



Total Test 



Achievement standard— 85% of the students in the province who wrote the test are 
expected to achieve the acceptable standard 



Figure 4-2 
Grade 9 Science 

Percentage of Students Achieving Standard of Excellence on the 
Total Test and on Components of the Test 

June 1993 




Concepts 



Skills 



Total Test 



Achievement standard— 15% of the students in the province who wrote the test are 
expected to achieve the standard of excellence 



Earlier studies on the language of 
testing show that the standards 
developed for the English version 
of the test cannot be directly 
applied to French Immersion 
students writing the translation; 
standards for the French 



Immersion and Francophone 
students were not set separately 
due to the small numbers involved. 
Thus, these students are not 
included in any discussion of the 
numbers of students achieving 
standards. 
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Table 4-3 
Grade 9 Science 

Students Achieving Standards on the Total Test and on Components of the Test 

June 1993 







Provincial 


Provincial 


Students Achieving Assessment j 




Maximum l Assessment 


Achievement 




Standard 






{ Possible ' 


Standard* 


Standard** 


Expected 


Actual 


Actual 


Reporting Category 


Score 


(Raw Score) 


(Percent) 


Number 


Number 


Percent 


Standard of Excellence 


i ' 












Total Test 


; 65 : 


52 


15 


4 292 


4 880 


17.1 


Skills Only 


29 


23 


15 


4 292 


6 184 


21.5 


Concepts Only 


! 36 


29 


15 

1 


4 292 


5 526 


19.3 


Acceptable Standard 















Total Test 


65 


33 


85 


24 321 


22 214 


77.6 


Skills Only 


29 


15 


85 


24 321 


21 413 


74.8 


Concepts Only 


! 36 


18 


85 


24 321 


22 987 


80.3 


Below Acceptable Standard 


j 1 
! N/A 


N/A 


N/A 


N/A 


4 214 


14.7 


on Both Skills and Concepts 


1 
1 








i 





* The Provincial Assessment Standard is a score determined by standard-setting procedures and is the lowest score a student must 
achieve for his/her performance to be judged "acceptable" and/or "excellent" in relation to curricular expectations. 
See Appendix A. 

** The Provincial Achievement Standard refers to the percentage of students expected to achieve the Provincial Assessment Standard. 



Tt should be noted that the actual 
percentages of students achieving 
standards on the total test for 
Grade 9 Science (17.1% and 77.6%) 
are based on the 28 613 students in 
the regular programs who wrote 
the test. 

If percentages are based on the 
total June population reported by 
principals (not including the 
French Immersion and 
Francophone students who wrote 
the test), the percentages achieving 
standards would be: 

14.9% achieving the standard of 

excellence, 

67.7% achieving the acceptable 
standard. 

If the percentages are based on the 
September 30, 1992, Grade 9 
enrollment (33 655), not including 
the French Immersion und 
Francophone students who wrote 



the test, the percentages achieving 
standards on the total test would 
be: 

14.5% achieving the standard of 
excellence, 

66.0% achieving the acceptable 
standard. 

It is emphasized that the 
percentages based on enrollment 
present the lowest estimate of 
achievement. It is highly likely 
that some of the students who were 
absent, exempt, or not accounted 
for could have achie*. ed standards. 

The number of students achieving 
the acceptable standard and the 
standard of excellence for each 
school was analyzed to determine 
whether achievement in schools 
was below, meeting, or exceeding 
provincial achievement standards. 
Schools classified as meeting 
provincial achievement standards 



were those for which the difference 
between the actual number of 
students and the expected number 
of students achieving standards 
was not statistically significant. 
Differences are only reported when 
there is a 5% or smaller probability 
that a difference of that size could 
occur by chance. 

As can be seen in Table 4-4, about 
one-third of schools were 
significantly below the provincial 
achievement standard. School 
administrators and teachers 
should critically examine science 
achievement test results to 
determine what they are going to 
do to improve the level of student 
achievement. Although this is true 
for all schools, it is particularly 
important for those schools 
performing below the provincial 
achievement standard. 
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Table 4-4 
Grade 9 Science 

Percentage Distribution of Schools* Meeting Achievement Standards on the Total Test 
(N=568) 

June 1993 



Standard 


Significantly Below 
Provincial Achievement 
Standard 


voiuayc WIBU IWUUVII VI IIS is 

Not Significantly Different 

From Provincial 
Achievement Standard 


Significantly Above 
Provincial Achievement 
Standard 


Standard of Excellence 


7.0 


77\b' 


15.3 


I 

\ Acceptable Standard 

i 


33.6 


63.6 


2.8 



'Schools with fewer than five students are excluded, as the statistical significance of the difference between the number actually achieving the 
standard and the number expected to achieve the standard when calculated and reported is not educationally meaningful. 



Reporting Categories 

Table 4-5 shows the total marks 
possible and the provincial raw 
score results for the reporting 
categories of the Grade 9 Science 
Achievement Test. 

It is important to stress that the 
averages on the various reporting 



categories cannot be directly 
compared with one another. 
Rather, the results shown in Table 
4-5 can best be used in conjunction 
with parallel tables in the 
jurisdiction and school reports. 
Variations in patterns of students' 
responses to questions can help to 
indicate strengths and weaknesses 



in local educational programs. 
Statistics presented in this table 
are based on results achieved by 
26 653 (students who wrote the 
regular form and those who wrote 
Form T). 



Table 4-5 
Grade 9 Science 

Raw Score Results by Reporting Category 

June 1993 



Reporting Category 



Number 
of Questions 



Topic 



Average Score 



Standard Deviation 



Diversity of Living Things 


: 12 


7.1 j 


2.3 


Fluids and Pressure 




7.7 


2.5 


Heat Energy: Transfer and Conservation 




4.3 


1.6 


Electromagnetic Systems 


; 


7.2 


2.2 


Chemical Properties and Changes 


i 11 


7.5 


2.3 


Environmental Quality 




7.3 


2.4 


Learning Domain 








Skills 


i 29 


18.2 


5.1 


Concepts 


! 36 

i 


22.9 


6.0 
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Results for Individual 
Multiple-Choice and 
Numerical-Response 
Questions 

Table 4-6 shows the percentage of 
students who chose each alternative 
(A, B, C, aiul D) for each multiple- 
choice question. The correct 



response (key) for each question is 
also identified. Table 4-7 shows the 
distribution of responses for each 
numerical-response question. The 
results shown in these tables can 
best be used in conjunction with 
the parallel tables in the 
jurisdiction and school reports. 
Variations in patterns of students' 



responses to questions can help to 
indicate strengths and weaknesses 
in local educational programs. 

Statistics presented in these tables 
are based on results achieved by 
26 653 students (those who wrote 
the regular form and those who 
wrote Form T). 



Table 4 6 
Grade 9 Science 

Results for Individual Multiple-Choice Questions 

June 1993 



Rem 



Distribution of Responses (%) 



B 



Topic: Diversity of Living Things 



Domain 



Curriculum Standard 



1 
5 

10 
11 

12 
14 

56 j 



4 
6 

52 



29.2 
18.9 



7.5 
74.7* 

47.8* 

7.9 
4.2 



2 83.1* 

3 j 46.2* 



12.6 
4.1 

25.4 
7.6 

9.2 
19.7 
48.1 

5.0 
8.6 



3.1 j 8.6 
59.8* ! 19.2 



I 



5.0 



5.5 



2.9 
3.2 

24.3 
15.8 

34.1 
15.5 
31.1* 

5.3 
6.1 

16.3 
17.1 

68.3* 



55.2* 
73.7* 

42.7* 

2.0 

8.8 
56.9* 
16.5 

6.5 
38.9 

72.0* 

3.9 

21.2 



Concepts 
Concepts 

Concepts 
Concepts 

Concepts 
Concepts 
Concepts 

Skills 
Skills 

Skills 
Skills 

Skills 



define the meaning of "species" 

know which type of plant structure best retains moisture 

know taxonomic levels used in scientific classification 

know how natural selection protects an insect population from 
changes in the environment 

identify types of specialization within a group of insects 

identify problems with selective breeding 

identify a benefit to society of an application of 
selective breeding 

infer the relatedness of relationships between organisms 

infer the relatedness of species on the basis of their 
classification 

classify oak trees using a dichotomous key 
infer the adaption of birds.to a food source 

predict the habitat that an organism has adapted to based on 
observed characteristics 



•correct answer 



Continued 
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Table 4-6 (continued) 



r 


Distribution of Responses (%) | 


Topic: Fluids and Pressure 


ItAm 


A 1 


] 

B | 


] 

c | 


j 

D | 


! 

Domain 


Curriculum Standard 


7 j 


73.6* 


21.8 


f 

3.4 j 


1.2 


Concepts 


know that increased density results in increased buoyancy 


; . ! 


5.8 


73.3* 

I 


I 

7.5 


13.3 

i 


Concepts 


know how pressure affects fluids in a siphon 


9 

j 1 


0.9 


1.4 ! 

I 


79.0* : 


18.6 


Concepts 


know that pressure decreases as surface area increases 


! 34 j 

; i 




46 ! 


i 

X* ft 


I 

CM O* 

o*t.o 




MIUW uicJ I clcUlUl lol lip Ucmccll IIUIU viouuoiiy diiu 

temperature change 


! 48 1 


10./ 


4o.o 


o.o 


OC Q 
^O.O 


ooncepis 


know how valves control the movement of fluids in a pump 


j ; 

j 49 

i i 


10.1 


21.8 


62.7* 


5.2 


Concepts 


know the effect of resistance on objects moving in fluids 


i 51 i 

; 
i 


23.7 


48.1* 


10.2 


17.9 


Concepts 


understand the relationship between buoyant force and the 
force of gravity 


58 


8.5 


60.7* 


22.0 


8.7 


Concepts 


know how valves control the movement of fluids 


32 

i 


11.4 


70.1* 




4.0 


14.3 


Skills 


infer that the reduced volume of a gas is related to the spaces 
between particles 


] 35 

! 


29.3 


60.6* 


6.0 


4.0 


Skills 


predict the effect of temperature changes on the viscosity of 
motor oil 


i 

| 36 

... 


51.5* 


21.3 


5.8 


21.3 


Skills 


apply knowledge of the relationship between pressure, force, 
and area in a hydraulic system 



' 1 

1 j Distribution of Responses (%) 


Topic: Heat Energy - Transfer and Conservation 


Item ' 

i j 

; j. 


A 


B 


C 


D 


Domain 


Curriculum Standard 


: 15 ! 

: i 


25.3* 


55.0 


2.5 


17.1 


Concepts 


know the effect of structural designs on the amount of cooling 
in houses 


; i 
j 16 j 


11.8 


80.5* 


5.8 


1.9 


Concepts 


know how a solar collector converts the sun's energy into heat 


i 17 j 


28.7 


41.6* 


13.7 


15.9 


Concepts 


know that metals are poor insulators 


18 


5.6 


11.9 


58.3* 


24.2 


Skills 


relate the effect of variables when conducting an experiment 


i 45 


8.1 


16.1 


54.6* 


21.2 


Skills 


judge the effects of heat transfer between solids and liquids 


I « 


14.1 


33.1* 


27.1 


25.7 


Skills 


infer the relationship between rate of temperature change and 
the mass of a fluid 


; 50 


83.2* 


4.7 


! 5.9 


6.0 


Skills 


i 

| apply the principle of specific heat capacity 

i 



*correct answer Continued 
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Table 4-6 (continued) 



Distribution of Responses (%) \ Topic: Electromagnetic Systems 



Item • 

i 


i 

a ; 


b ; 


C 


D I 


I 

Domain 


Curriculum Standard ] 


19 ! 


7.4 i 

; 


4.6 I 


65.7* j 


22.2 


Concepts ; 


know the purpose of a thermocouple 


39 i 

! 


79.2* ! 


3.8 j 


4.2 

i 


7.7 j 


Concepts 


know that changing electric current direction affects the 
direction of a compass needle j 


40 ; 


5.1 


4.4 


i 

11.0 

! 
i 


79.4* ! 


Concepts 


understand how the strength of an electromagnet affects the 
speed of an electric motor 


59 ! 


6.2 


8.5 


74.5' 


10.7 


Concepts 


know how a bimetal strip can interrupt an electric circuit 


60 I 

u 


12.0 


"»2.8 


16.7 

4 


58.4* 


Concepts 


know the function of a fuse in a circuit 


41 ! 


26.0 


14.5 


44.3* 


14.4 


Skills 


predict a hidden circuit connection from circuit test results 


42 I 

J 


60.6* 


22.5 


13.3 


3.6 


Skills 


interpret information in a graph to show the relationship of 
resistance and electric current 


j 


3.9 


4.8 


86.7* 


4.6 


Skills 


apply knowledge of how switches work in a circuit 


44 I 

i 

i 


50.9* 


21.3 


7.0 


20.7 


Skills 


predict the effect of changing a circuit that has electric loads 
linked in series and parallel 


47 j 

I 
i 

L i. 


8.3 


78.5* 


i 

| 6.2 

! 


6.9 


Skills 


predict the effect on temperature caused by a resistor in a 
circuit 



Distribution of Responses (%) i Topic: Chemical Properties and Changes 



| Item 

i 




B 




D 


Domain 


Curriculum Standard 


j 20 


14.5 | 


70.9* 


7.3 ! 

■ j 


7.3 


Concepts 


identify a chemical change in different experimental results 


! 21 


5.2 j 


70.7* 


! 15.1 ! 

! ! 


8.8 


Concepts 


understand the effect of surface area on reaction rates 


i 23 


3.9 


17.4 


. 2.4 j 


76.3* 


Concepts 


identify the presence of acids and bases in household products 


I 24 


11.1 j 


21.2 


: i 
! 13.0 I 

! | 


54.6* 


< 

Concepts 


know that the physical property density can be used to classify 
different materials 


! 33 


4.1 


14.8 


3.3 j 

i i 


77.8* 


Concepts 

i 


know a method of preventing corrosion 


' 38 


15.2 ; 


4.7 


; 3.6 | 


76.5* 


I Concepts 


understand that chemical change in materials produces heat 




69.6* ! 


21.3 


3.5 ! 

| 


5.6 


j Concepts 

; 

i 


understand that heating a solution causes an increase 
in solubility 

I 


22 


7.5 


73.3* 


7.7 i 


11 5 


Skills 


; organize data that will show the effecl of acid on a material 


« 37 


1.8 j 


3.9 


! t 
84.9* | 


9.4 


i 

| Skills 


| interpret information from a diagram and apply understanding 



of melting point 



•correct answer Continued 
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Table 4-6 (continued) 





Distribution of Responses (%) 


Topic: Environmental Quality 


Item 


I 

A ' 


B 


> i 
j C 1 


D 


Domain 


Curriculum Standard 


13 

i 


64.0* ; 


14.1 


' 11.9 I 


9.9 


Concepts 


know the negative impact of chemical use on the environment 


28 

1 


12.2 I 

i 


12.5 


■ 3 


57.6* 


Concepts 


identify the cause of low levels of dissolved oxygen in 
river water 


i 29 

i 


: 

8.8 I 

! 


7.2 


j 73.7* | 

i 


10.2 


Concepts 


know a procedure that could be used to measure the effect of 
chemical material on fish living in a river 


j ■ 30 


2.5 ! 


3.2 


j 86.0* ! 


8.2 


Concepts 


identify an example of recycling used tires 


; 53 

i 


9.9 


4.6 


11.6 

i 


73.8* 


Concepts 

I 1 


identify an environmentally friendly action in response to an 
environmental problem 


j £X> 

i 


14.3 


17.6 


~1 

i 61.2* 


6.7 


Skills 


interpret data and identify correct experimental procedure 


| 26 


71.6* 


11.0 


j 

! 14.9 

I 


2.4 


Skills 


calculate the average num 1 of organisms per drop of water 


! 27 

i 


35.3* 

j 


11.9 


| 40.3 


12.4 


Skills 


interpret data by relating the average population of an 
organism to the amount of water analyzed 


! 31 


j 

j 7.8 

i 


14.6 


| 72.1* 

I 


5.4 


Skills 

i 


infer the cause of increased phosphate and nitrate levels 
in a river 


j 54 

i 

■ 


I 9.0 

j 


67.1* 

I 


| 10.1 

i 


13.7 


1 

I Skills 

i 


analyze data to find an increase of toxin levels in higher levels 
of a food pyramid 


! 55 
I 


t 64.4 

I 

i 


! 18.8 

i 


! 10.1 

i 


6.6 


Skills 


predict the effect of thermo pollution on health of trout living 
in a lake 




i . JL 


i 









•correct answer 



Table 4-7 
Grade 9 Science 

Results for Individual Numerical-Response Questions 

June 1993 



I 



Distribution of 
Responses (%) 



"1 



Correct 



i Item i Response 



1 



44.6 



Incorrect | No i 
Response i Response 



55.1 



72.1 | 27.3 



0.3 



0.6 



Topic 

Electromagnetic 
Systems 

Fluids and 
Pressure 



57.9 i 41.4 j 0.7 j Heat Energy 



37.6 ' 60.8 



54.8 39.3 



1.6 



5.9 



I 

; Chemical 
1 Properties 

Chemical 
Properties 



Domain 

Concepts 

Skills 

Skills 
Skills 



Skills 



Correct Answer 
4231 



Curriculum Standard 

describe effect of resistance 
on electron flow 



predict changes in buoyant j 4213 

force that result from changes in j 
fluid density 

interpret data and compare 3142 

heat conduction rates of materials \ 

infer and identify changes in « 1 122 

materials as chemical or physical j 

calculate the density based on 1 .75 

mass and displacement ; 
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Examiner's Observations 

Generally, the Grade 9 teachers 
who reviewed and set standards for 
the test felt that it was a good 
reflection of the Grade 9 Science 
program. They felt that the theme 
layout provided an excellent 
method for organizing the 



questions and giving real-world 
context to each question. There 
was a good range of question types 
and difficulties that were 
representative of the science, 
technology, and society emphasis of 
the program. The use of numerical- 
response questions gave more 



information about what students 
can do in science. 

Sample questions and commentary 
are provided. The correct answer is 
marked with an asterisk, and the 
percentages of students choosing 
each alternative is shown. 

Acceptable Standard 

For question 11, students had to 
know that natural selection 
protects an insect population from 
changes in the environment. 
Almost four out of five students 
achieving the acceptable standard 
know this. 

For question 37, students were 
required to interpret information 
from a graph and apply 
understanding of melting point of a 
substance. Most students can do 
this. 

To successfully answer 
numerical-response question 2, 

students had to predict the effect of 
density on buoyant force in four 
sugar solutions. Nearly four out 
five students achieving the 
acceptable standard can do this. 

For question 18^ students had to 
identify the controlled variables of 
an experiment. Students achieving 
the acceptable standard but not 
the standard of excellence had 
difficulty doing this. 



Use the following information to answer question 11. 



i Over the past 20 years, the farmer kept careful records of different insect populations 
! on his farm. His findings and observations are as follows: 

i 

! • Many insect species exhibit several distinct adult forms within the same species 
| (polymorphism). The farmer noted that each form has adapted for a particular 
function. 

• From 1976 to 1989, the farmer found that he needed to use a higher concentration 
of insecticide each year to control the grasshopper population. 

• In 1987, the farmer discovered aphids in his clover crop. Aphidt, are tiny insects 
that use the juices from plants for food. Frequently, aphids can cause a reduction 
in plant growth. 



1 1 . The most probable reason that the farmer needed to use a higher concentration of 
insecticide each year from 1976 to 1989 is that the insects 

74.7 *A. that survived previous applications passed their resistance on to their offspring 
7.6 B. were not affected by the insecticide because it was absorbed into the soil, water, 

and air 

15.8 C. reproduced quickly enough to replace those kilied by the insecticide 
2.0 D. increased in size and weight 



Use the following information to answer question 37. 

[ 

| George plans to use a paste wax to protect the paint cn his car. He designs an 
experiment to test for the melting point of the wax. 

i X represents a sample of the same wax in each test tube. 



37. The melting point of the wax is 

1.8 A. lower than 25°C 

3.9 B. between 25°C and 60°C 
84.9 *C. between 60°C and 90°C 

9.4 D. higher than 90°C 



ERIC 
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Use the following information to answer numerical-response question 2. 

! A candy maker tested four different sugar solutions by placing a hydrometer in each. 




12 3 4 



2. Arrange the four solutions in order from lowest density to highest density. 
72.1 Answer: 4,2,1,3 



Use the following information to answer question 18. 

i 

j Mr. and Mrs. Brown decide to use curtain material that will make the 
| house more energy efficient. They experiment to determine the best 
; solar energy-absorbing material. They note the following variables: 

I. time of day 

II. type of material 

III. length of time in the sunlight 

IV. surface area exposed to sunlight 



18. The variables that should be kept constant in this experiment are 

5.6 A. I and II 
11.9 B. II and IV 
58.3 *C. UH.andlV 
24.2 D. II, ill, and IV 

Use the following information to answer question 53. 



: Jane reports to the group. She tells them that acid occurs when products of j 

! combustion of fossil fuels are released into the atmosphere, combine with j 

j moisture, and fall to the earth as acid rain or snow. The high acidity damages j 

] both water environments and human-made structures. In an effort to combat j 

! the effects of acid rain, governments have begun dumping large quantities of J 

, calcium carbonate (limestone) into lakes to reduce their acidity. j 



53. For an environmentalist, which would be the most environmentally friendly means of 
dealing with the problem of acid rain? 

9.9 A. Reducing the acidity with a base 

4.6 B. Putting acid-tolerant fish into the lakes 

1 1 .6 C. Adding calcium carbonate (limestone) to the lakes 

73.8 *D. Reducing the amount of sulpf ijr oxides released into the atmosphere 



Overall, results show that the 
students who achieved the 
acceptable standard but not the 
standard, of excellence, have an 
ability to 

• know the effects of increased 
surface area 

• know that natural selection 
protects populations (question 11) 

• understand the effects of reaction 
rates 

• interpret 'information from a 
diagram (question 37) 

• know that electric currents cause 
magnetic fields 

• predict temperature changes 
caused by a resistor in a circuit 

• apply the principle of specific heat 
capacity 

• identify environmentally friendly 
actions 

• know that a bimetal strip can 
interrupt an electric circuit 

• predict the effect of changing 
density on buoyant force 
(numerical-response 2) 

Many of the these students ; 
however, did not do as well as 
expected in 

• defining the meaning of species 

• identifying controlled variables in 
an experiment (question 18) 

• identifying causes of reduced 
oxygen levels in a river 

• knowing the purpose of a 
thermocouple 

• interpreting a graph that shows 
an inverse relationship 

• predicting the effects of changes 
to parallel and series circuits 

• interpreting heat conduction data 

• using higher level skills 

• applying skills in novel situations 



62 
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Use the following information to answer question 58. 



During open heart surgery, doctors often use artificial valves. If a person has 
a heart with a defective valve, an artificial valve can be used to replace the 
defective valve. This is illustrated below. 



Left 
»tnum 



*i/tJ* AntficuJ 



Bill type jnifKuJ vilve 



SiU lp open 
position 



Lea 

vewnde 



Bili m doted 
position 



58. When the ball of the valve is in the "open" position, blood is 

8.5 A. forced out of the left ventricle into the aorta 
60.7 *B. supplied by the left atrium to the left ventricle 
22.0 C. forced out of the left ventricle into the left atrium 

8.7 D. supplied by the aorta to the left ventricle 

Use the following information to answer numerical-response question 5. 

r 

| A crime lab analyst use J displacement to calculate the density of a piece of 
I metal. The mass of the metal is 70.0g. When the metal was added to the 
I beaker, the displaced water overflowed into the cylinder. 



5. Calculate the density of the metal in g/cm 3 . DO NOT ROUND YOUR ANSWER. 

54.8 Answer: 1.75 g/cm 3 

Use the following information to answer question 41. 



! Sandy finds an old rail switching circuit board that does not have a circuit 
| diagram. When she tests this circuit board using a dry cell and bulb, she 
! obtains the following results: 



Contacts 
Tested 


T 

! HI 


H J 


HK I HL 


HM 


I J 


IK ] I L i JK 


! 5 


Response i + : 

i of Bu,b L 


+ ! - 


+ 


— 


i 

+ ! " 




*! -i 



+ Bulb lights 

— Bulb doos not light 



Standard of Excellence 

The following commentary 
highlights the skills and 
knowledge of students who 
achieved the standard of excellence. 

For question 53, students had to 
identify an environmentally 
friendly action in response to an 
environmental problem. Nearly 
95% of students achieving the 
standard of excellence were 
successful on this question. 

For question 58, students needed 
to know how valves control the 
movement of fluids. About nine out 
of ten students achieving this 
standard know this. 

Numerical-response question 5 

required students to calculate the 
density of a metal based on 
measured mass and volume 
displacement. Nearly all of these 
students can do this. 

Question 41 required students to 
predict hidden circuit connections 
from circuit test data. Some 
students achieving the standard of 
excellence had difficulty doing this. 



41. Tho bulb will also light for connection 



26.0 
14.5 
44.9 
14.4 



ERLC 



A. JM 

B. JL 
»C. IM 
D. KL 
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Students who achieved the 
standard of excellence 
demonstrated more success than 
did other students when answering 
questions that required applying 
science concepts in novel or new 
contexts, identifying variables in 
experiments, and interpreting 
complex information or data. 
Specifically, students who achieved 
the standard of excellence could 

• comprehend the meaning of 
scientific and technological terms 

• infer relatedness of species 

• classify organisms 

• identify variables in experiments 

• interpret information and data 
about the environment 

• identify the purpose of different 
structures of living things 



• identify problems with selective 
breeding 

• predict the effects of temperature 
changes on liquid viscosity 

• predict habitat of organisms 

• evaluate environmental actions 
(question 53) 

• analyze how valves control the 
movement of fluids (question 58) 

• know the purpose of an electrical 
device 

• interpret information about 
chemical and physical changes 

• interpret information about heat 
conduction 

• calculate density (numerical- 
response 5) 



However, these students did not do 
as well as expected in 

• interpreting information about 
heat conduction and radiation 

• selecting a physical property that 
would classify materials in a 
group 

• understanding the application of 
average population of organisms 

• interpreting circuit diagrams and 
charts (question 41) 

• identifying the manipulated 
variable on a graph 

• identifying the benefit of a 
medical development on society 

• understanding the inverse 
relationship between resistance 
and conductor diameter an., "or 
temperature. 



t < 
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Performance-Based 

Assessment 

General Description 

A total of 697 randomly selected 
students from a sample of schools 
throughout the province were 
involved in performance-based 
assessment. 

Performance-based assessment 
was developed to assess students' 
higher order thinking skills in real- 
life problem-solving situations. 



The tasks assessed aspects of 
science that could not be measured 
adequately by paper-and-pencil 
tests in which only the answer is 
recorded and marked. 

Students were asked to solve six 
real-life problems and were given 
concrete materials and/or 
information to solve these 
problems. These activities are 
described in Table 4-8. 
A group of experienced Grade 9 



Science teachers met in July 1993 
and established scoring standards. 
All student responses were scored 
following the standards set, with 
about 20% of papers being rescored 
to ensure marker consistency. 
Markers used holistic scales to rate 
student responses for problem 
solving and communication. 

The results are shown in Table 4-9 
and figures 4-3 and 4-4. 



Table 4-8 
Grade 9 Science 

Performance-Based Assessment Activities 



Activity 



1 



Name 



Seed Dispersal 



Sugar Solution 



Sleeping Bags 



Robot Arm 



Bulk Food Store 



Hazardous Waste 



Topic 



Program Area 



Diversity of Living Things 



Fluids and Pressure 



Heat Energy 



Nature of Science 



j Science and Technology 



Science and Technology 



Electromagnetic Systems j Science and Technology 



Chemical Properties 
Environmental Quality 



Nature of Science 



Science, Technology, and 
Society 



Learner Expectation 



Observe seeds 
Infer seed dispersal 
Predict dispersal patterns 

Construct and calibrate a hydrometer 
Use a hydrometer to measure liquid 
density 

Design and execute an experiment to test 
insulation effectiveness 

Design, test, troubteshoot, evaluate, and 
construct an electromagnet 

Develop an experimental procedure to 
identify white powders 

Analyze information 



i Examine perspectives 
I Identify alternatives 
j Consider consequences 
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Table 4-9 
Grade 9 Science 

Performance-Based Assessment Results 



•S > 
So 

CLCO 





Activity 1 


Activity 2 


Activity 3 


Activity 4 . 


Activity 5 


Activity 6 


Student Achievement 


(%) 


(%) 


(%) 


(%) 


(%) 


(%) 


Beyond Grade 9 


15.5 


27.7 


28.0 


13.7 


23.0 


2.2 


At Grade 9 


44.0 


16.6 


48.5 


24.9 


43.2 


27.6 


Not Yet at Grade 9 


29.5 


30.5 


19.1 


27.2 


25.9 


! 64.4 


Totally Misunderstood or 


11.0 


25.2 


4.4 


34.2 


7.9 


j 5.8 


Left Blank 








i 




i 


Beyond Grade 9 


24.8 


17.9 


25.0 


10.0 


18.6 


4.1 


At Grade 9 


39.0 


21.4 


49.2 


25.9 


39.7 


26.8 


Not Yet at Grade 9 


24.9 


: 26.9 


20.4 


30.4 


31.5 


63.0 


Unclear and 


11.3 


33.8 


5.4 


33.7 


10.2 




Inappropriate 




i 
{ 
! 








| 
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Figure 4-3 
Grade 9 Science 

Percentage of Students Achieving At or Beyond Grade 9 Expectation (PBA) 9 by Activity 
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Activity 2 Activity 3 
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Figure 4-4 
Grade 9 Science 

Percentage of Students Achieving At or Beyond Grade 9 Expectation (PBA), by Number of Activities 
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Examiner's Observations 

The results show students could 

• solve problems using scientific 
inquiry skills 

• make simple observations and 
inferences 

• use a variety of strategies when 
given more materials than 
needed to solve a problem 

• account for the manipulated 
variable and responding variable 
in an experiment 

• get the correct solution if they 
could construct a device to solve 
the problem 

• arrive at the correct solution if 
they understood the problem and 
implemented an appropriate 
strategy 

The results show that students 
found it difficult to 

• solve open-ended problems 
successfully 



• suggest more than one alternative 
or perspective on an issue 

• construct a device to solve a 
practical problem 

• look at an environmental issue 
from different perspectives 

• make detailed observations 

• account for controlled variables 
in an experiment 

• identify and communicate the 
relationships found in a set of 
data or observations 

• communicate in forms other 
than — or in addition to — writing 
in sentences and paragraphs 

There are gender similarities and 
differences in students' ability to 
solve problems; girls were better 
than boys in collecting and 
interpreting data and applying 
decision-making processes in an 
environmental issue. Boys were 
better at constructing devices to 
solve problems. Both were 



successful at applying basic 
inquiry skills in problem 
solving. 

Overall, these results indicate that 
Grade 9 students have weak 
inquiry and problem-solving skills. 
They attained acceptable 
performances on only two of the six 
PBA activities. It is apparent that 
the new Grade 9 science program 
has not been fully implemented in 
Alberta schools and that many 
students are not receiving 
sufficient experiences in hands-on 
inquiry, hands-on problem solving, 
and environmental decision 
making. 

A more detailed description of the 
assessment tasks and scoring 
criteria, and samples of students' 
performances are provided in the 
V oklet Samples of Students 1 
Responses from the June 1993 
Grade 9 Science Performance- 
Based Assessment Tasks. 



Relationship between 
Performance-Based 
Assessment and the 
Achievement Test 

Generally, results show a positive 
correlation between achievement 
test s r ores and scores on the 
performance-based assessment. 
The highest correlations occur 
between the total achievement test 
score and each of the following 
performance -based activities: Seed 
Dispersal (0.50), Sugar Solutions 
(0.43), and Robot Arm (0.42). Also, 
positive but lower correlations 
occur for the other three activities: 
Sleeping Bags (0.27), Bulk Food 



Store (0.29), and Hazardous Waste 
(0.20). Students had greater 
difficulty demonstrating problem- 
solving skills in the performance- 
based assessment tasks than on 
the machine -scored component. 

Also, other possible relationships 
between student results on the 
Grade 9 Performance-based 
Assessment and student 
performance on the Grade 9 
Science Achievement Test were 
explored by classifying students 
according to their performance on 
the achievement test. Students 
were split into three groups: 
students achieving the standard of 

SO 



excellence, students achieving the 
acceptable standard but not the 
standard of excellence, and students 
not yet achieving the acceptable 
standard. Table 4-10 contains the 
results of this analysis. 

Students achieving the standard of 
excellence on the achievement test 
also performed higher than other 
students on all performance 
assessment tasks. However, only 
2.5% of these students attained a 
level of excellence for Activity 6 — 
Hazardous Waste, the most difficult 
task. 
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Table 4 10 
Grade 9 Science 

Relationship Between Achievement Test Results and 
Performance-Based Assessment 



j Achievement Test Results (Percentage) j 


\ Problem-Solving Component 


Communications Component j 


Performance-Based ; 


Standard of 


j 


Below Acceptable 


Standard of 




Below Acceptable 


Assessment Score i 

. 1 — 


Excellence 


i Acceptable Standard i 


Standard . 


Excellence 


Acceptable Standard 


Standard 


0 ! 


4.5 


j 8.6 


29.5 


4.5 


7.7 


30.7 


1 ; 


13.6 


! 37.7 ! 


50.0 


11.6 


31.8 


37.5 


2 


52.5 


! 44.5 


20.5 


41.9 


43.2 


25.0 


3 i 




Q 1 

y.i 

— : 1 


n n 




17.3 


6.8 

j 


u 


14 ft 


C.O.C. 


□u.u 


1Q 9 


32.7 


67.0 


; 1 


20.7 


; 37.7 


37.5 


24.2 


32.3 


21.6 


' 2 J 


20.2 


18.2 


6.8 


29.3 


22.3 


10.2 


! 3 


44.4 


20.9 


5.7 


27.3 


12.7 


1.1 


i i 
l n '' 


2.0 


; 3.2 


8.0 


2.5 


4.5 


9.1 


i 

: 1 j 


13.6 


; 21.8 


31.8 


14.1 


21.4 


36.4 


2 ! 


48.0 


51.4 


40.9 


45.5 


56.4 


38.6 


3 


36.4 


23.6 


19.3 


37.9 


17.7 


15.9 


! 0 


15.2 


37.7 


j 56.8 


15.2 


38.2 


54.5 | 


; 1 


24.2 


; 30.5 


31.8 


25.3 


33.2 


33.0 


! 2 


35.4 


: 21.8 


| 9.1 


38.4 


23.2 


11.4 


I 3 I 


25.3 


: 10.0 


! 2.3 

! 


21.2 


5.5 


1.1 

i 


i 5 ! 


3.0 


9.5 


! 11.4 


3.5 


11.3 


18.2 




17.7 


29.1 


46.6 


28.3 


32.3 


42.0 


i 2 j 


47.5 


; 43.2 


| 29.5 


41 .4 


41.8 


30.7 


; 3 


31.8 


18.2 


j 12.5 


26.8 


14.1 


9.1 


: 0 


3.0 


6.8 


| 3.4 


2.5 


6.9 


5.7 


j 

»: 1 i 


59.6 


! 64.8 


I 86.4 


53.0 


66.1 


87.5 


2 


34.8 


25.1 


j 10.2 


37.9 


22.9 


6.8 


3 


2.5 


j 3.2 


; 0.0 


i 6.6 


4.1 


0.0 



< 



CO 

> 1 



Overall, students achieving the 
acceptable standard and not the 
standard of excellence on the 
achievement test found the 
performance-based tasks 
significantly more challenging than 
students who achieved the standard 
of excellence. They were able to 
respond best to the Sleeping Bags 
activity; however, only 28.3% 
achieved the acceptable standard on 
the Hazardous Waste activity. This 
highlights the difficulty students 
have addressing environmental 
issues from different perspectives. 

Students achieving below the 
acceptable standard had a low 
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success rate on five of the six 
activities. They had the greatest 
success on Activity 3 — Sleeping 
Bags, where 60.2% and 54.5% of 
students achieved at or beyond 
Grade 9 expectations on problem- 
solving and communication skills, 
respectively. They had the least 
success with the Hazardous Waste 
activity, where 10.2% of these 
students achieved the acceptable 
standard for problem-solving skills. 

Contexts for Learning 
General Description 

In June 1993, 154 Grade 9 Science 
teachers and 532 students from 
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across the province participated in 
a study designed to examine 
relationships among various 
contexts for learning and student 
achievement. Students responded 
to questions related to the 
attitudes outlined in the Program 
of Studies for Science. Teachers 
reported on the types and 
frequency of use of instructional 
strategies, activities, classroom 
resources, and manipulatives. 
Results of this study are reported 
in Table 4-11 and Table 4-12. 



Table 4 11 
Grade 9 Science 

Contexts for Learning: Percentage Distribution of Student Responses 



Statement 


Always 1 


Often 


i 

Sometimes i 


Rarely 


i Never ; 


NO ; 

Comment j 


1 . There are many ways to find an 
answer to a practical problem in 

science. \ 

j 


2.3 

i 


53.0 


22.0 j 

i 

i 


2.0 


! <1 - 0 ! 

i I 

; ! 


<1.0 

I 
i 

i 


c.. \ am coniiosni inai wncn i am 
presented with a problem in 
science, i can usually iina a way to 
- solve it. 


*7 n 

! 

j 


ft 

\ 


QO ft 


ft ft 
o.u 


1ft i 

! 

; i 

i ; 


1 0 ! 

i 
\ 


3. It is important to know some 
| science in order to get a good job. 


36.0 


42.0 

! 


18.0 


2.0 


| <1.0 

i 


<1.0 


; 4. 1 am good at science 


8.0 


38.0 j 


40.0 


8.0 


I 3.0 

j 


3.0 


i 

i 5. Science helps me: . 

i a. understand how things work 

b. prepare for high school 

c. solve practical problems 

j d. make decisions(env. qual) 


34.0 
38.0 
24.0 
27.0 


46.0 j 
35.0 ! 
45.0 j 
37.0 \ 


17.0 
19.0 
22.0 
26.0 


2.0 
5.0 

5.0 

a ft 
b.U 


' <i.o 

j <1.0 
i 2.0 
I 2.U 


<1.0 
3.0 

1.0 

■i ft 

l .u 


I 6. Science is useful for solving every 
j day problems. 


15.0 


36.0 | 

j 


34.0 


12.0 


j 1.0 


1.0 


! 7. 1 enjoy doing science experiments 


31.0 


31.0 i 

> I 


26.0 


8.0 

i 


; 2.0 


1.0 


1 o. i iiKe io De cnauengeu m science 


i 

i OC\ ft 
j c\).\J 


i Oft ft i 


0.1 ft 


i 

I o.u 
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^ ft 
o.u 


1 9. 1 feel successful in science 


I 7.0 


| 33.0 'j 


42.0 


! 12.0 


3.0 


3.0 


! 10. The more I learn in science, the 
j more interesting it becomes. 


! 24.0 

I 


! 38.0 ' 

i 


25.0 


I 

| 7.0 


! 3.0 


I 2.0 

1 
1 


! 11. I like to be presented with a 
l problem in science, and then 
develop my own procedure to 
: solve it. 


12.0 

j 


I ; 
j 25.0 j 

i i 

1 


35.0 


! 17.0 


I 9.0 

i 


j 2.0 


i 12. I would rather watch the teacher do 
a lab demonstration than do the 
investigation myself. 


j 6.0 


; 9.0 


23.0 


29.0 


31.0 

! 

! 


1.0 

i 


13. Theories in science cannot be 
questioned or changed. 


8.0 


; 13.0 


31.0 


19.0 


14.0 


i 14.0 

I 


14. If, after solving a challenging 
\ science problem. I found that other 
classmates had different results, I 
would: 

a. assume I made a mistake and 
that my results are incorrect 

b. assume the others made a 
mistake 

c. want to share my results with 
others 


; 5.0 
2.0 
' 22.0 


18.0* 
11.0 

■ 38.0 

i 


44.0 
42.0 
24.0 


23.0 
j 30.0 
8.0 


7.0 
11.0 
4.0 


i 

2.0 

| 3.0 

I 

3.0 


15. I like to work in groups in science 


54.0 


28.0 


13.0 


2.0 


2.0 


1.0 



class. 
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Table 4-11 (continued) 



.Statement Always 

16. I enjoy doing science projects 



Often | Sometimes i Rarely 



24.0 



35.0 



27.0 



9.0 



17. I have used the following sources 
of information this year to gain 
information for science research 
reports or science projects: 

a. experts 

b. references 

c. community resources 

18. I do science experiments at home. 

19. If a conclusion is going to have 
merit, then the experiment can he 
repeated and the same results will 
be generated. 

20. I am careful to record exactly what 30.0 ; 42.0 
I observe so that my conclusions ! 
are based on the most accurate 
data. j 

21. Technological problems are solved 8.0 20.0 
by using the same methods or 
procedures each time. 

22. I find it easy to work with 20.0 j 32.0 
technological equipment 

23. Environmental problems can be 15.0 47.0 
solved with technological or 
scientific knowledge 

24. If I have a viewpoint on an 14.0 i 23.0. 
environmental issue, ! will try to 
convince a person who has an 
opposite viewpoint that they are 
wrong. 

25. When I am using a piece of new 26.0 , 33.0 
technological or scientific 
equipment, I study or try to find 

\ out the correct safety procedures 
for its use. 



Never 



2.0 



No I 
Comment 



21.0 



39.0 



31.0 



28.0 



28.0 



27.0 



3.0 



19.0 



10.0 



3.0 



22.0 



5.0 



1.0 



5.0 



3.0 



2.0 



9.0 



4.0 



2.0 



7.0 


13.0 


26.0 


i 

: 23.0 


25.0 


6.0 


20.0 


31.0 


28.0 


j 11.0 


4.0 


4.0 


4.0 


13.0 


25.0 


| 28.0 


22.0 


6.0 


2.0 


3.0 


22.0 


i 

j 33.0 

i 


36.0 


! 3.0 


15.0 


23.0 


34.0 


8.0 


1.0 


17.0 



1.0 



8.0 



3.0 



6.0 



5.0 



4.0 
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Table 4-12 
Grade 9 Science 

Contents for Learning: Percentage Distribution of Teacher Responses 



Statement 



Never 



Rarely 



Sometimes Often 



I used the following instructional 
strategies with my Science 9 students 
this year: 

a Full class discussion 

b. Reading from a texibook 

c. Worksheets 

d. Small Group discussion 

e. Research 

f. Oral presentations 
g Project work 

h. Viewing films/video 

i. Lecture and note taking 
j. Lab investigations 

k. Demonstrations 
I. Field Trips 

How often did your grade 9 Science 
class do the following kinds of writing 
this year? 
a Journal entries 

b. Essays 

c. Summaries 

d. Descriptions 

e. Note taking 

f. Ideas from research 

g Ideas from group/class discussion 
h. Lab write-ups 



J. 



Reports 

Generating questions 



k. Inquiry activity write-ups 



This year, my students have 
presented projects in the following 
formats: 
a 



Written 
Story 
Essay 
Display 
Videotape 
Newspaper article 
Poster series 



0.0 
12 
00 
1.7 
12 
13.5 
1.8 
4.7 
12 
0.0 
0.0 
39.5 



69.8 
23.3 
2.9 
2.4 
0.6 
5.3 
1.7 
0.0 
3.5 
8.1 
42 



2.3 
58.7 
29.5 
13.1 
77.0 
61.1 
25.4 



1.8 
4.3 
20.6 
22.1 
25.7 
33.9 
18.8 
28.5 
13.4 
1.7 
52 
442 



13.6 
46.5 
20.6 
11.3 
8.7 
23.7 
16.3 
2.9 
282 
22.7 
13.7 



7.0 
23.4 
28.3 
25.2 
10.9 
20.4 
20.1 



Always 



N/A 



32.7 
42.4 
47.6 
49.4 
52.9 
43.9 
54.1 
552 
43.6 
23.3 
40.1 
11.6 



8.3 

24,4 

42.9 

49.4 

372 

50.9 

48.8 

20.9 

47.1 

39.5 

36.3 



43.3 
12.6 
31.9 
42.3 
,.3 
12.0 
42.0 



53.8 
395 
23.8 
24.4 
13.5 

7.6 
22.9 
11.0 
36.6 
56.4 
44.8 

1.7 



5.9 
47 
29,4 
31,5 
43.6 
17.8 
28.5 
52.3 
17,1 
22.7 
33.3 



38.0 
1.8 
72 
13.7 
1.8 
3.0 
10.7 



11.7 
7.6 
2.9 
2.3 
1.8 
1.6 
2.4 
0.6 
52 
18.6 
9.9 
12 



1.8 
0.6 
3.5 
42 
9.9 
2.4 
4.7 
23.8 
3.5 
6.4 
10.7 



9.4 
0.6 
1.8 
1.8 
0.6 
1.2 
1.8 



0.0 
0.0 
0.0 
0.0 
0.0 
0.0 
0.0 
0.0 
0.0 
0.0 
0.0 
1.7 



0.6 
0.6 
0.6 
12 
0.0 
0.0 

0.0 
0.6 
0.6 
1.8 



0.0 
3.0 
12 
0.0 
2.4 
2.4 
0.0 
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Table 4-12 (continued) 



Statement 


| Never 


j Rarely 

j 


' Sometimes 

J ■ 


Often 


I Always 


N/A 


h. Audiotape 


! 84.9 


j 8.4 


! 1.8 


1.8 


! 0.0 


3.0 


i. Scale model 


I 44.3 


21.0 


i 263 

j 


42 


! « 


3.0 


j. Play 


j 74.7 


I 127 


8.4 

i 


0.6 


i 0.6 


3.0 


k. Slides and Commentary 


j 83.6 


! 9.1 

I 


4.8 


0.0 


0.0 


24 


1. Working model 


| 228 


21.6 

; 


! 41.9 


10.8 


! ^ 
i 


1.8 


1 encourage a variety of formats for 
presenting research reports 


| 4.7 


! 

i 129 


i 

! 

; 421 


24.0 


; 11.7 


4:7 



5. I encourage students to design their 
own experiments 

6. I demonstrated that practical 
problems may have a variety of 
viable solutions 

7. How often did your students have a 

; choice about how they would design 
\ their investigations? 

• 8. My students participate in at least one 
i "hands-on" activity in science each 
I week 

| 9. My students do at least one lab activity 
I in science each week. 

1 0. Through modelling and discussion, 
my students are encouraged to 
realize that a particular point of view 
must be supported with factual data. 

1 1 . When my students were presented 
with an open-ended problem where 
they were expected to develop their 
own plan, most of them appeared: 
a Motivated 

b. Bored 

c. Frustrated 

d. Confident 

e. Insecure 



23 



0.0 



23 



0.0 



0.0 



0.0 



i 12 

j 1>9 

| 12 

| 8.9 

i 

! 



15.7 



1.8 



221 



5.8 



3.5 



28 
421 
17.9 
16.8 
31.6 



48.3 



25.1 



552 



28.1 



222 



26.3 



392 
41.3 
615 
46.0 
44.3 



25.6 



532 



14.0 



25.1 



292 



392 



122 
7.5 
17.9 
342 
152 



8.1 
19.9 

5.8 

415 

42.7 
31.0 



8.4 



12 



0.0 



0.0 



0.6 



0.0 



0.0 



0.0 



12 

1.9 
0.6 
0.6 
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Table 4-12 (continued) 



I 

Statement 


Yes . 


No 


N/A j 


; 12 Which of the following types of 

i assessment did you use with your ' 

\ science siuQenis mis year: 

a Multiple choice tests : 


j 

96.8 


12 | 


_ , 

i 
i 

: 
i 

— 


\ b. Short answer i 


994 


0.6 


— 


| c. True-false 


84.9 j 


13.4 


1.7 


d Extended response ■ 


94.8 


29 ! 


23 


\ e. Performance assessment ' 


703 I 

i 


25.0 ! 


4.7 


! f. Lab write up i 


92.7 


23 


_ 


i . ' 

, y» r icotJI llclUUI lo 


71.5 


27.9 


0.6 


h. Checklists 


372 1 


59.9 


29 


i. Anecdotal records 


355 


622 


23 


j. Student self-evaluation 


42.4 


57.0 


0.6 


k. Vocabulary match 


79.1 


19.8 


12 


I. Interpreting diagrams etc. 


99.4 


0.0 


0.6 


m. Creating diagrams, etc. 
\ and/or modeling to show 
I understanding 


805 


16.5 


3.0 


| a Problem solving 


97.7 


1.7 


' 0.6 


I 

I 

i 13. Were the follow»~ig activities a pari 
' of the grade 9 science program this 
year? 

a Field trips 


36.0 


62.8 


12 


b. Science fair 


i 26.7 


715 


1.7 


c. Science Olympics 


19.8 


78.5 


1.7 


I d. Listening to guest speakers 


372 


| 61.6 


12 


j e. Decision making 

i 


93.0 


5.8 


12 


f. Writing assignments 


95.9 


| 3.5 


0.6 


[ g. Completing handouts 


I 942 


i 

: 5.8 

1 


i 
i 


• h. Watching demonstrations 


j 98.8 




t _ 

i 


j i. Lab activities/experiments 


100.0 


I 

i 


i 


| j. Research projects 


I 87B 

I 


11.6 


0.6 


■ k. Making models 


j 72.1 


| 26.7 


12 


I. Participating in debates 


| 48.8 


| 48.8 


! 23 


J m. Problem solving/small group 


! 91.3 


i 6.4 


! *3 


I a Problem solving w/ STS 


i 88.4 


105 


i 12 


■ o. Open-ended investigations 


! 84.3 


! 14.0 


| 1.7 


p. Analyzing photo/graph/table 


, 98.3 


1.7 


I 
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Table 4-12 (continued) 



Statement 



Yes 



No 



N/A 



q. Using a computer 

r. Writing frequent quizzes 

s. Working in small groups 

t Writing in a journal 

a Completing exercises 

v. Generating solutions 

w. Evaluating designs 

x. Taking action toward an issue 

y. Evaluating effects of action 



22.1 
72.1 
95.3 
17.4 
97.7 
95.9 
80.8 
49.4 
59.3 



75.6 
27.3 
3.5 
802 
1.7 
23 
192 
49.4 
38.4 



23 
0.6 
12 
23 
0.6 
1.7 

23 
23 



14. This year, the following items were 
readily available in the science 
room for my students or my use in 
planning and instructing: 
a Science Directions 9 TRM 

b. Science Directions 9 ST 

c. Sci + Tech. & Soc. TRM 

d. Sc. + Tech. & Soc. ST 

e. Water Lit. Prog. -AB. Env. 

f. Space Age Agr. Res. 

g. Energ. Lit. Series(SEEDS) 

h. Aquatic Invert. Mon. Prog. 

i. Tables or lab counters 
j. Sinks for lab stations 
k. Films/videos 

I. Equipped 

m. Basic Mat. for Topic 1 

n. Basic Mat. for Topic 2 

o. Basic Mat. for Topic 3 

p. Basic Mat. for Topic 4 

q. Basic Mat. for Topic 5 

r. Basic Mat. for Topic 6 



97.1 

97.1 

45.3 

36.6 

33.1 

11.0 

326 

16.3 

91.9 

87.8 

85.5 

88.4 

93.0 

924 

93.6 

88.4 

93.6' 

802 



I. 7 
23 
523 
622 
66.3 
86.6 
64.5 
81.4 
7.6 

II. 0 
13.4 
8.1 
4.1 
4.1 
3.5 
8.7 
29 
16.3 



12 

0.6 
23 
12 
0.6 
23 
29 
23 
0.6 
12 
12 
3.5 
29 
3.5 
29 
29 
3.5 
3.5 
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Student Attitude 
Questionnaire 

The student questionnaires reveal 
that students geue rally feel that: 

• it is important to know some 
science in order to prepare for 
high school and to get a good job 

• science helps them understand 
how things work and to solve 
problems 

• it is important to record exactly 
what they observe so that their 
conclusions are based on the 
most accurate data 

Relationships between Student 
Attitudes and the Results on 
the Achievement Test 

Students achieving the standard of 
excellence on the achievement test 
understand that science helps 
them solve problems. They 
indicated that science will help 
them get ready for high school and 
to get a job. This group of students 
reported greater curiosity about 
events and objects in the natural 
world. They were more likely to be 
open-minded and thought that 
critical thinking was important. 
As a group, they also enjoyed doing 
experiments. 

Of the students achieving the 
acceptable standard, but not the 
standard of excellence, only 36.6% 
indicated they were often good at 
science. Also, only 56.1% indicated 
they often enjoyed doing 
experiments and 55% often enjoyed 
doing projects. However, this group 
of students did report a tendency to 
respect accuracy and precision, be 
critically minded and appreciate 
the contribution of science for 
solving problems. Most of these 
students liked to work in groups. 
Many of the students achieving 



below the acceptable standard 
indicated that learning science 
would help them get a job and that 
honesty in reporting is important. 
Only 23.3% of these students 
thought they were good at science. 
Also, only 58.7% of students in this 
group indicated they often enjoyed 
doing experiments in science. As a 
group, they showed a lower 
tendency to be critically minded. 

Teacher Questionnaire 

The teacher questionnaire reveals 
that teachers reported 

• using a variety of instructional 
strategies with their Science 9 
students, with full class 
discussions and lab 
investigations the most 
frequently mentioned strategy. 

• having students do various 
writing, with note taking, lab 
write-ups, and inquiry activity 
write-ups the most frequently 
mentioned activities. 

• having students do at least one 
investigation per week. 

• demonstrating to students that 
practical problems can have 
more than one solution. 

• using a variety of assessment 
methods. 

Journal writing was not being 
emphasized in schools — 69.8% of 
teachers never had students write 
in a journal. This is of interest, 
because journal writing can be 
used for student self-assessment. 

Students tended to present their 
projects in written form (90.7%); 
however, they rarely write stories 
(only 15%) or essays (only 40.9%). 
In the future, there may be an 
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opportunity for teachers to explore 
language communication in 
science. 

Teachers used a variety of 
assessment types, which tend to be 
teacher directed. Student self- 
evaluation was reported to be used 
by only 42.4% of the teachers. 

Enrichment activities were not 
emphasized — for example, only 
36.0% of teachers reported having 
their classes go on field trips, and 
only 37.2% of teachers reported 
having students listen to guest 
speakers. Enrichment activities 
can provide opportunities for 
students to see Science, 
Technology, and Society 
connections. 

Only 48.8% of teachers reported 
having their students participate 
in debates. Debates can provide an 
excellent opportunity for students 
to appreciate the different sides to 
a STS issue. Also, only 49.4% of 
teachers reported having students 
take action toward an STS issue. 

Although there is a technology 
focus in the Grade 9 & 'ence 
program, only 22.1% of the 
teachers reported using computers 
for instruction, 

Relationships between Context 
for Learning and the Results 
on the Achievement Test 

Possible relationships between 
reported teacher behaviours and 
student performance on the 1993 
Grade 9 Science Achievement Test 
were explored by classifying 
students according to their 
performance and looking for 
differences in the frequency of 
responses on the teacher survey. 
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Students were split into three 
groups: students achieving the 
standard of excellence, students 
achieving the acceptable standard 
but not the standard of excellence, 
and students not yet achieving the 
acceptable standard. Student 
achievement results for 532 
students were matched with 
survey responses of their teachers 
(154 teachers). 

Teaching behaviours as reported 
by teachers of students achieving 
the standard of excellence were 
compared to the responses of 
teachers of students who had 
achieved the acceptable standard 
and to the responses of teachers of 
students who had not yet achieved 
the acceptable standard. 

For students achieving the 
standard of excellence, 93% of their 
teachers reported emphasizing 
technological problem solving and 
societal decision making, whereas 
for students achieving below the 



acceptable standard only 73.9% of 
their teachers indicated 
emphasizing these skills in their 
class. Also, for students achieving 
the standard of excellence 7G.1% of 
their teachers indicated that they 
encouraged students to support 
their points of view with factual 
data compared to 43.4% for 
teachers of students who had not 
achieved the acceptable standard. 

For students achieving the 
standard of excellence, their 
teachers reported having students 
write about generating questions 
(43.7%), write about generating 
ideas from discussion (50.7%), and 
write up inquiry activities (81.7%) 
more frequently than the teachers 
of students who had not yet 
achieved the acceptable standard 
(8.6%, 13.0%, and 47.8% 
respectively). 

For students who had achieved the 
standard of excellence, their 
teachers indicated they placed a 



greater emphasis on problem 
solving (93.0%) than the teachers 
cf students who had not achieved 
the acceptable standard (73.9%). 
Also, for students who had 
achieved the standard of excellence 
52. 1% of their teachers indicated a 
preference for having students do 
self-assessment compared to 39.1% 
of the teachers of students who had 
not achieved the acceptable 
standard. 

Students who had achieved the 
standard of excellence had teachers 
who indicated a greater preference 
to supplement their programs with 
additional instructional materials 
than teachers of students who had 
not achieved the acceptable 
standard. Also, students who had 
achieved the standard of excellence 
had teachers who indicated a 
greater usage of computers (28.2%) 
than the teachers of students who 
had not achieved the acceptable 
standard (4.3%). 
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Section 5 
Achievement by Gender 

v 



Information on the gender of 
students who wrote the provincial 
achievement tests has been 
collected, analyzed, and reported 
since 1989. 

This section of the report answers 
the following questions: 

• What is the proportion of males 
and females who wrote the 1993 
achievement tests? 

• Is the percentage of males and 
females meeting standards the 
same in each subject? 

• Are the 1993 results for 
individual subjects similar to or 
different from those of 1989 
through 1992? 



Figure 5-1 

Number of Achievement Tests Written by Gender 

June' 1993 
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Observations and 
Discussion 

What is the proportion of 
males and females who wrote 
the 1993 achievement tests? 



Results for 1993, presented in 
Figure 5-1, reveal that more males 
than females wrote the 
achievement tests at each grade 
level. This pattern is similar to the 
data from 1989 through 1992, with 
the exception of 1990 when more 
females than males wrote the 
Grade 9 English Language Arts 
test. 
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Is the percentage of males and 
females meeting standards the 
same in each subject? 

Results for 1993, presented in 
Figures 5-2 and 5-3, reveal that 
more females than males achieved 
the acceptable standard and the 
standard of excellence in Grade 
Level 3 Language Learning and 
Grade 6 Social Studies. In Grade 9 
Science, however, males 
outperformed females, with higher 
percentages achieving both the 
acceptable standard and the 
standard of excellence. 

Are the 1993 results for 
individual subjects similar to 
or different from those of 1989 
through 1992? 

The 1993 pattern of results for 
Grade Level 3 Language Learning 
is similar to the Grade 3 results 
from 1989. A similar pattern was 
found in both the 1990 Grade 9 
and 1992 Grade 6 English 
Language Arts results. 

The pattern for the 1993 Grade 6 
Social Studies results is similar to 
the 1989 Grade 6 and the 1992 
Grade 3 Social Studies results: 
females outperformed males at 
both the acceptable standard and 
the standard of excellence. In 1991, 
however, males marginally 
outperformed females in Grade 9 
Social Studies. 

The 1993 Grade 9 Science results 
show a similar pattern to the 1989 
Grade 9 and the 1990 Grade 6 
Science results: males 
outperformed females, particularly 
at the standard of excellence. In 
1991, however, males and females 
showed similar levels of 
achievement in Grade 3 Science. 

There are gender differences in the 
levels of achievement attained by 



Figure 5-2 

Number and Percentage of Students Achieving Acceptable 
Standard on the Total Test by Gender 

June 1993 
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* 85% of students were expected to achieve the acceptable standard on the total test. 
Figure 5-3 

Number and Percentage of Students Achieving Standard of 
Excellence on the Total Test by Gender 

June 1993 
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males and females on the 
provincial achievement tests in the 
three grades and subjects tested in 
1993. Since individual jurisdiction 
and school results may show 
patterns that differ from the 
province-wide results, school 
personnel are encouraged to 



explore gender differences within 
their own settings. 

We welcome any comments 
regarding observations or thoughts 
on gender differences in 
achievement. 
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Section 6 



Achievement by Age 



Information on the age of 
students who wrote the provincial 
achievement tests has been 
collected, analyzed, and reported 
since 1990. 

This section of the report answers 
the following questions: 



• What is the age distribution of 
students who wrote the 1993 
achievement tests? 

• What relationship, if any, does 
a^e have with achievement as 
measured by the 1993 provincial 
achievement tests? 



Age Groups 

When achievement tests were 
written in June 1993, students' 
ages in both years and months 
were collected. Based on this 
information, each student was 
categorized as being in the first or 
the last half of his or her age year 



Table 6-1 

Number of Achievement Tests Written by Age* 

June 1993 
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*Ago on the test day (June 8, 1 993) 

An empty area indicates that there were no students 
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( e.g., 7 years = 84 to 89 months, 
7.5 years = 90 to 95 months). Table 
6-1 indicates the number and 
percentage of achievement tests 
written by age group and shows 
the exact range, in months, used to 
determine these data. Figure 6-1 
and Figure 6-2 show the proportion 
of students in each age group who 
achieved standards in each grade. 



Figure 6-1 

Percentage of Students Achieving Acceptable Standard on the 
Total Test by Age 

June 1993 
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Figure 6-2 

Percentage of Students Achieving Standard of Excellence on the 
Total Test by Age 

June 1993 
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Observations 
and Discussion 

What is the age distribution of 
students who wrote the 1993 
achievement tests? 

Of the Grade Level 3 students 
tested in 1993, 42.5% were 8 years 
olds and 51.4% were 9 years old at 
the time of testing. Almost 5% of 
the students tested were 10 years 
old or more. Thirty-five of these 
students were aged 12, 13 or 14. 
Sixty-eight students were reported 
to be 7 years old or younger. 

In 1993, 40.7% of tested Grade 6 
students were 11 years old at the 
time of testing and 51.8% were 12 
years old. Of the tested students, 
fewer than 1% were younger than 
11 years old and 6.5% were older 
than 12 at the time of writing. 
Eighty students who wrote the 
Grade 6 test were 14 or 15 years 
old on that day. 

For the year 1993, 42.3% of tested 
Grade 9 students were 14 years old 
on June 8 and 49.9% were 15 years 
old. Of the students tested, 0.3% 
were younger than 14 years old on 
the day of testing and 7% were 
older than 15. Ninety-four students 
were 17 years or older on the day 
of testing. 

While the number of students 
writing the Grade Level 3 test who 
were reported to be 7 years old and 
younger v/as relatively small, this 
figure may, nevertheless, be 
spuriously high due to reporting 
error. Caution, therefore, is 
advised when making inferences 
about participation rates and 
levels of achiever mt for students 
in this age range. 



What relationship, if any, does 
age have with achievement as 
measured by the 1993 
provincial achievement tests? 

In Grade Level 3, proportionally 
fewer students who were older 
than 9.5 years achieved the 
acceptable standard or the 
standard of excellence. 

On the Grade 6 test, 
proportionately fewer students 
aged 12.5 years and older achieved 
the acceptable standard and the 
standard of excellence. 

For Grade 9, there is an overall 
negative relationship between age 
and student achievement. Younger 
students generally achieve higher 
than do older students with regard 
to both the acceptable standard 
and the standard of excellence. 

These data indicate that, in 
general, students who are older 
than the median age for their 
grade did not achieve as well as did 
younger students who wrote the 
test. There is a considerable drop 
in the proportion of students 
meeting standards among those 
whose age is at the upper rang( of 
the mainstream and beyond. 
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Section 7 



Achievement by 
Grade Level 



This section answers the following 
question: 

What percentage of grades 3, 
6, and 9 Alberta students 

• may be achieving beyond 
grade level 

• are achieving at grade level 

• are not yet achieving at grade 
level 

as measured by the June 1993 
achievement tests? 

Within each curriculum, specific 
outcomes, expressed as the 
knowledge, skills, and attitudes to 
be acquired and developed, are 
arranged into sequences reflecting 
the developmental nature of 
learning and expectations for a 
learner's increasing competence 
in the subject area. 

The 1993 achievement tests were 
designed to measure student 
achievement within the given 
grade level. However, a number of 
questions on the achievement tests 
could be considered to measure 
more advanced levels of 
performance. For reporting 
purposes, we have classified 
students within the grade levels 
framework based on the following 
three definitions: 



2. Achieving At Grade Level 

Students who are achieving at 
grade level are those students 
who achieved the acceptable 
standard on the total test and 
those students who achieved the 
. standard of excellence on the 
total test but not on both 
components. 

3. Not Yet Achieving At Grade 
Level 

Students who are not yet 
achieving at grade level are 
those students who did not 
achieve the acceptable standard 
on the total test. 

This classification is a different 
way of viewing the results from 
the achievement test, which 
means that the numbers 
reported differ from those 
indicated for percentage of 
students achieving standards. 

Results from the 1993 
achievement tests showing 
distributions of grade level 
achievement are presented 
in Figure 7-1. This analysis 
shows the wide range of student 
achievement within a nominal 
grade level. 



May Be Achieving Beyond 
Grade Level 

Students who may be achieving 
beyond grade level are those 
students who achieved the 
standard of excellence for both 
major components of the test. 
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Figure 7-1 

Distribution of Grade Luvel Achievement 

June 1993 
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Section H - 
Achievement Over Time 

V 



An important task of Alberta 
Education is to measure and report 
changes in student achievement 
overtime. Since 1983, the Student 
Evaluation Bran :h has been 
measuring the achievement of 
students in grades 3, 6, and 9 in 
the subject areas o\ language arts, 
mathematics, science, and social 
studies. The administration of the 
Achievement Testing Program 
follows a four-year cycle for each 
subject within each grade level. 
In 1985, 1989, and 1993, students 
in Grade Level 3 Language 
Learning, Grade 6 Social Studies 
and Grade 9 Science were tested. 

This section of the report answers 
the following question: 

• Has achievement, as measured 
by individual provincial 
achievement tests, changed since 
1985 and 1989? 

Two studies were carried out. The 
first was designed to compare 
achievement since 1985 on the 
multiple-choice questions in Grade 
Level 3 Language Learning, Grade 
6 Social Studies, and Grade 9 
Science. The second study was 



designed to compare achievement 
since 1985 and 1989 on the written- 
response part of the Grade Level 3 
Langu Learning test and the 
Grade 6 Social Studies test. 

Descriptions of the designs and 
results of these special studies 
follow. 



Multiple-Choice Study 
Design and Methodology 

A random sample of students was 
selected, by school, in each of grades 
3, 6, and 9, to participate i * he 
achievement-over-time stu> . T n 
each grade, the achievemex. srer- 
time (AOT) sample was divia: 1 into 
three sub-groups. At each grade 
level, one sub-group, referred to as 
the AOT93 group, wrote the regular 
1993 achievement test. Students in 
the remaining sub-groups wrote one 
or the other of two AOT test forms 
that were specifically designed for 
this study. These clusters of 
students are referred to as the 
AOT89 group or the AOT85 group, 
depending on which AOT test form 
they wrote. 



Sampling of the schools involved in 
the study was carried out 
independently in grades 3, 6, and 
9. Only schools having 18 or more 
students in regular English 
programs in 1992, in the grade 
being sampled, were included in 
the groups ^om which samples 
were drawn. The sample sizes were 
targeted for approximately 3 000 
students at each grade level. Since 
the sampling was carried out 
independently at each grade level, 
some schools were randomly 
selected to participate at more 
than one grade. Students within 
each class were randomly assigned, 
by the Student Evaluation Branch, 
to write either the AOT93, the 
AOT89, or the AOT85 test form. 

In each grade, the 1989 and 1985 
AOT test forms included questions 
taken from the 1993 achievement 
test for that grade and portions of 
the original achievement test from 
1989 or 1985. Table 8-1 shows the 
composition of these, and the 1993 
AOT test form, for each grade. 
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Table 8-1 

Design of the Achievement-Over-Time Tests 

Number of Questions from Previous and Current Tests 





1993 


1989 


1985 




AOT Test 


AOT Test 


AOT Test 


Grade Level 3 Language Learning 








Number of 1 985 Questions 


0 


0 


21 


Number of 1989 Questions 


0 


21 


0 


Number of 1 993 Questions 


40 


19 


19 


Grade 6 Social Studies 








Number of 1985 Questions 


0 


0 


28 


Number of 1989 Questions 


0 


30 


0 


Number of 1993 Questions* 


47 


20 


22 


Grade 9 Science 








Number of 1 985 Questions 


0 


0 


23 


Number of 1989 Questions 


0 


20 


0 


Number of 1993 Questions 


65 


45 


42 



•Three questions on the 1993 Grade 6 Social Studies Achievement Test were dropped. These questions were dropped from the AOT93 form 
and did not appear on either AOT89 or AOT85. 



The 1989 items on the AOT89 form 
were used to compare 1993 student 
performance to student 
performance on these items in 
1989. The 1985 items on the AOT85 
form performed a parallel function. 
Before these comparisons were 
undertaken, however, it was 
necessary to verify that the 
students selected to participate in 
the special study were 
representative of students 
throughout the province. This was 
achieved by comparing AOT 
student performance on the 1993 
items with provincial performance 
on the same items. The 1993 items 
on the AOT forrr also allowed for 
the calculation ot a 1993 test score 
for each of the students in the AOT 
study. 

Experience has shown that schools 
participating in the AOT study, like 
mopt other schools, prepare their 
students to write achievement tests 
by administering earlier versions of 



those tests as practice. Questions 
from these earlier versions are the 
same ones used in the AOT study. 
Thus, scores for AOT students 
could be spuriously high due to the 
effects of practice. To check this 
possibility, a questionnaire was 
sent to teachers of the AOT groups 
to elicit information about student 
preparation practices. Based on 
teacher responses to the practice 
effects questionnaire, AOT 
students were divided into 
"practice" and "no practice" groups 
depending on whether or not they 
were reported to have used either 
1989 or 1985 tests in preparation 
for writing the 1993 achievement 
test. Group performance on the 
1993 items was compared to 
determine if the "practice" and "no 
practice" groups were of equal 
ability. Their performance on the 
1989 (or 1985) items was then 
compared to determine if there 
were differences that could be 
attributed to the effects of practice. 



Following the check for 
representativeness and the effects 
of practice, we were able to 
proceed with the performance 
comparisons of 1993 AOT students 
and students from 1989 and 1985. 
When making these comparisons, 
student results from 1989 and 
1985 were excluded from the 
analysis if those students were 
from schools that were smaller 
than the smallest school taking 
part in the 1993 AOT study. 

All comparisons made in this 
study (i.e., to check for 
representativeness and the effects 
of practice, and to make 
judgements about achievement 
over time) were assessed using 
statistical tests of significance. 
Because these tests are very 
sensitive when used with large 
samples, as was the case in this 
study, only differences having a 
probability less than 0.01 were 
considered to be significant. 
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Results 

Table 8-2 presents a comparison of 
AOT sub-group results and 1993 
provincial results. The purpose of 



this comparison was to verify that 
the students selected to participate 
in +he special study were 



representative of students 
throughout the province. 



Table 8-2 

Comparison of AOT Sub-group Results with 1993 Provincial Results* 





AOT 93 


Province 


AOT 89 


Province 


AOT 85 


Province 




1993 Items 


1993 Items 


1993 Items 


1993 Items 


1993 Items 


1993 Items 


Grade Level 3 Language Learning 














Number of 1993 Questions 


40 


40 


19 


19 


19 


19 


Raw Score Mean 


26.3 


26.8 


13.4" 


13.0 


13.3 


13.2 


Standard Deviation 


7.9 


7.9 


3.8 


3.9 


4.1 


4.0 


Number of Students 


959 


33 546 


935 


33 546 


938 


33 546 


Grade 6 Social Studies 














Number of 1993 Questions 


47 


47 


20 


20 


22 


22 


Raw Score Mean 


29.8 


29.9 


12.3 


12.3 


12.8 


13.0 


Standard Deviation 


7.2 




3.2 


3.3 


3.8 


3.8 


Number of Students 


938 


28 W 


932 


28 987 


935 


28 987 


Grade 9 Science 














Number of 1993 Questions 


65 


65 


45 


45 


42 


42 


Raw Score Mean 


40.4 


41.2 


27.7** 


28.7 


25.8 * 


26.5 


Standard Deviation 


10.6 


10.4 


7.6 


7.4 


. 7.3 


7.1 


Number of Students 


986 


25 178 


977 


25 178 


972 


25 178 



•Students from schools smaller than the smallest school taking part in the AOT study are excluded from this analysis. Provincial data in this 
table, therefore, differ from those reported elsewhere in this report. 

**On these items, the mean of the AOT group is significantly different from the provincial mean. 



The data indicate that there were 
no significant differences between 
provincial means and the means of 
the Grade 6 Social Studies AOT89 
and AOT85 groups, or between 
provincial means and the Grade 
Level 3 Language Learning AOT85 
means. There were, however, 
significant differences between the 
provincial mean and the AOT89 
group in Grade 3. As well, in Grade 
9, both the AOT89 and the AOT85 
group means differed significantly 
from the provincial means on the 
same grouping of items. This 



indicates that the Grade 6 Social 
Studies sample and the Grade 3 
AOT85 sample were representative 
of the province. However, the Grade 
3 1989 AOT group and the Grade 9 
1989 and 1985 AOT groups were 
not. To address this issue of non- 
representativeness, test equating 
procedures were used when 
comparing 1993 Grade 3 
performance with that in 1989 and 
when comparing 1993 Grade 9 
performance with that from 1989 
and 1985. 



The AOT comparison data is 
provided in Table 8-3. Included in 
this table, for each grade, is the 
number of questions on which 
comparisons were based, the mean 
and standard deviation for each 
year of comparison, and the 
number of students in each 
comparison group. It should be 
noted that equated means are 
reported for the Grade 3 1989 AOT 
group and the Grade 9 1989 and 
1985 AOT groups. 
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Table 8-3 

Achievement-Over-Time Comparison of Descriptive Statistics* 





AOT 89 Group 


1989 Province 


AOT 85 Group 


1985 Province 




(Wrote in 1993) 


(Wrote in 1989) 


(Wrote in 1993) 


(Wrote in 1985) 


Grade Level 3 Language Learning 










Number of Questions 


21 


21 


21 


21 


Raw Score Mean 


i6.r 


15.3 


15.3** 


14.6 


Standard Deviation 


4.4 


4.5 


4.0 


3.7 


Number of Students 


935 


. 31 014 


938 


30 976 


Grade 6 Social Studies 










Number of Questions 


30 


30 


28 


28 


Raw Score Mean 


20.2** 


19.4 


18.2** 


17.2 


Standard Deviation 


6.0 


5.8 


4.6 


4.8 


Number of Students 


932 


29 554 


935 


28 793 


Grade 9 Science 










Number of Questions 


20 


20 


23 


23 


Raw Score Mean 


13.3 


13.6 


13.6** 


14.0 


Standard Deviation 


4.0 


3.8 


4.1 


4.2 


Number of Students 


977 


27 041 


972 


29 513 



* Students from schools smaller than the smallest school taking part in the AOT study are excluded from this analysis. Provincial data from 
1989 and 1985 differ, therefore, from those reported in earlier reports. 

** On these items, *he mean of the AOT group is significantly different from the provincial mean. 



The data indicate that 1993 
students in Grade Level 3 and 
Grade 6 achieved significantly 
higher scores than did their cohorts 
from earlier years. In Grade 9 the 
trends were reversed, with the 
students from earlier years 
achieving the higher scores. 

Inferences about these differences 
should be drawn with caution, as 
the results of the practice-effects 
portion of the study indicate that 
some of the 1993 scores may be 
spuriously high due to the effect of 
students having prepared for the 
test using items from the 1989 and 
1985 achievement tests. In 
particular, there is some evidence 



to suggest that the Grade 3 AOT85 
and the Grade 6 AOT89 groups 
may have benefited from such 
practice. Consequently, it seems 
reasonable to infer that some, 
albeit not all, of the score increases 
in grades 3 and 6 are due to the 
effects of practice. 

The 1993 decline in Grade 9 scores, 
relative to those from 1989 and 
1985, does not seem to have been 
mitigated by the effects of practice: 
few Grade 9 AOT students were 
reported to have used previous 
tests in preparation for the 
achievement test, and those who 
did practice did not achieve higher 
scores. 



In conclusion, the data from the 
multiple-choice study indicate that 
achievement in Grade Level 3 
Language Learning and Grade 6 
Social Studies is somewhat higher 
than that in earlier years. Science 
achievement in Grade 9 does not 
seem to have fared as well, 
however, with 1993 students 
scoring about the same as students 
in 1989, but lower than those in 
1985. 
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Written-Response Study 
Grade Level 3 
Language Learning 
Design and Methodology 

Students' writing performance has 
been of particular interest to 
educators and to the public since 
the beginning of the Achievement 
Testing Program. Because of this 
interest, a study was initiated to 
compare 1993 writing with 1989 
writing and 1985 writing in Grade 
Level 3 Language Learning. This 
was a descriptive study that 
required teacher-readers to take a 
research or reader-as-observer look 
at the papers they read, rather 
than the usual evaluative or 
reader- as-assessor view of a 
teacher-marker. No attempt was 
made to rescore papers; rather, 
teacher-readers described features 
of the 1993, 1989, and 1985 writing 
in the three scoring categories: 

•Content and Development 
—relationship between events, 
actions, and/or ideas and the 
context established by the 
writer 
-beginning 
-specificity of detail 



-connections and/or 
relationships between events, 
actions, details, and or 
characters 

-ending 

-reader- writer relationship 

•Use of Language 
—degree to which standard 

sentence constructions are 

present and controlled 
-presence of different sentence 

patterns and length 
-quality of words and 

expressions 

•Conventions 
-end punctuation and 

capitalization 
-spelling 
-clarity 



Comparisons were made at two 
standards: Clearly Meets the 
Acceptable Standard (formerly 
Acceptable — 3), which represents 
work at an acceptable level for 
students completing Grade 3, and 
Clearly Meets the Standard of 
Excellence (formerly Excellent — 5), 
which represents outstandirg work 
for students completing Grade 3. 



Papers read in the study were 
selected at random from papers 
that received scores of Clearly Meets 
the Acceptable Standard or Clearly 
Meets Standard of Excellence on the 
June 1993, June 1989, and June 
1985 achievement tests. 

A group of experienced teacher- 
readers, representing all major 
regions of the province and a 
variety of school settings, reviewed 
the selected papers. 

Working alone, then in pairs, and 
then discussing the papers as a 
group, the teacher- readers 
described the papers for features of 
content and development, use of 
language, and conventions. 

They then compared their 
descriptions of 1993, 1989, 1985 
Clearly Meets the Acceptable 
Standard (3) and Clearly Beyond 
Standard of Excellence (5) papers 
to draw conclusions. 
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Results 

Clearly Meets the Acceptable 
Standard (3) 

Teacher-readers felt that papers 
judged to Clearly Meet the 
Acceptable Standard (3) in 1993 
were not significantly different 


from 1985 Acceptable (3). Although 
they thought that 1989 Acceptable 
(3) papers were in the range of the 
1993 papers Clearly Meeting the 
Acceptable Standard, they found 
both the 1993 and the 1985 papers 
to be generally superior to the 1989 


papers, particularly in content and 
development. Teacher-reader 
comments on key features of these 
papers are summarized below. 


Clearly Meets the Acceptable Standard (3) fn Grade Level 3 Language Learning 




Key Features of 1993 Papers 


Key Features of 1989 Papers 


i\ey reaiures or isoa capers 


Content and Development 


Content and Development 


Content and Development 


•events, actions, and/or ideas appropriate 
for context 


•only some events, actions, and/or ideas are 
appropriate for context 


•events actions, and/or ideas are 
appropriate for context 


•clear idea of storyline development 


•disjointed storylines 


•sequential development; simple recounting 
of events 


•strong beginnings 


•confusing beginnings 


•appropriate beginnings 


•middle consists of actions, (events), 
characters not directly related 


•events were rambling and connections to 
other events or characters were not evident 


•middle consists of actions, (events), 
characters not directly related 


•weak or missing transitions were common in all three years 
•appropriate but abrupt closures were common in all three years 


Use of Language 


Use of Language 


Use of Language 


•standard sentence constructions 
present 


•standard sentence constructions not 
usually present 


•standard sentence constructions present 


•variety of sentence length and type 


•simple sentences predominate 


•some variety of sentence length and type 


•specific words attempted 


•few vocabulary risks 


•fe\N vocabulary risks 


•simple subordination or coordination using (and/or) common in all three years 


Conventions 


Conventions 


Conventions 


•somewhat controlled (many run-on 
sentences 


•somewhat controlled 


•very controlled, confidently used 


•end punctuation: periods or 
exclam^.on marks 


•end punctuation: periods 


•end punctuation: periods 


•quotation marks attempted 


•quotation marks generally not attempted 


•quotation marks generally not attempted 


•spelling errors common; 
errors relatively phonetic 


•few spelling errors 


•few spelling errors 



Excellence (5) 

Teacher-readers ielt that papers 
judged to be clearly beyond the 
Standard of Excellence (5) in 1993 



were superior to the papers from 
1989 and 1985, which were, 
nonetheless, high quality but more 
similar to each other. A summary 



of the teacher-reader comments on 
differences and similarities in the 
three reporting categories follows. 



Clearly Beyond the Standard of Excellence (5) In Grade Level 3 Language Learning 



Key Features of 1993 Papers 


Key Features of 1989 Papers 


Key Features of 1985 Papers 


Content and Development 


Content and Development 


Content and Development 


•events, actions, and/or ideas consistently 
appropriate 


•appropriate ideas and details used to 
develop storylines 


•appropriate ideas and details used to 
develop storylines 


•beginning captures reader's attention; 
provides direction for writing 


•strong beginning 


•strong beginning 


•specific, effective details consistently used 


•specific details used 


•specific details used 


•connections/relationships between events, 
actions, details and/or characters 
consistently maintained 


•events and details were arranged in a 
purpose T ul, effective order 


•events and details were arranged in a 
purposeful, effective order 


•clearly wrote for an audience 


•audience not clear 


•audience not clear 


•strong, connected enJings 


•abrupt and/or contrived endings 


•abrupt and/or contrived endings 


•writer's voice was strong 


•writer's voice not evident 


•writer's voice ;iot evident 


•students in all three years captured reader interest by writing with imagination and creativity 


Use of Language 


Use of Language 


Use of Language 


•presentation consistently controlled 


•presentation usually controlled 


•presentation usually controlled 


•good variety of sentence type and length 


•some variety of sentence type and length 


•some variety of sentence type and length 


•words and expressions timely and 
consistently precise and effective 


•specific or descriptive verbs tended to be 
well used 


•specific or descriptive adjectives and verbs 
tended to be used well 


•willingness to take vocabulary risks with 
good results 


■willingness to take vocabulary risks, but 
these were not always successful 


•few vocabulary risks taken 


Conventions 


j Conventions 

•punctuation controlled in all three years 
•few spelling errors in all three years 


| Conventions 


Concluding Comments 

The question that guided this study 
was: Has writing by Grade Level 
3 Language Learning students 
improved from 1985 to 1993? 

Teachers participating in the study 
thought that in 1993 students 
clearly meeting the acceptable 
standard (3) produced writing of 
similar quality when compared to 


their 1985 and 1989 counterparts. 
However, in 1993 3tudents, whose 
work was clearly beyond the 
standard of excellence^ produced 
higher calibre writing when 
compared to their 1985 and 1989 
counterparts. 

Teacher-readers made the 
following observations related to 


the changing nature of the writing 
over the three test years. 

The different prompts used on the 
three tests may have influenced 
the type of writing produced by 
students. Teacher-readers felt that 
the 1985 prompt tended to be 
narrow in focus and did not allow 
the students enough leeway to 
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express themselves. The prompt in 
1989 was perhaps more open- 
ended, but much of the students' 
interpretation hinged upon their 
understanding of the word 
"fragile", which figured 
prominently on a large packing box 
that fell from a fast-moving truck. 

The teacher-readers all agreed that 
the 1993 "hats" prompt gave 
students the most latitude to 
develop their stories and that the 
picture of different hats, which 
accompanied the prompt, gave 
them possibilities from which to 
springboard their ideas. 

All believed that the 10 minutes of 
collaboration time given in 1993 



before writing commenced made a 
difference to the students, if not 
directly in their writing, at least in 
their comfort level. The fact that 
students were told to choose the 
format — story, letter, or series of 
diary/journal entries — that would 
allow them to do their best writing 
was also a positive change in the 
teachers' view. 

It was very obvious to the teachers 
that the students writing in 1993 
come from print-rich classroom 
environments where literature 
plays a key role. This strong 
literature base was very evident in 
the writing that these students 
produced. 



While not in the realm of 
assessment, teachers also noted 
that penmanship quality has 
deteriorated over the three test 
years. As well, in the writing from 
all three years, both themes and 
supporting details strongly 
reflected the popular media 
influences of the time. Storylines 
and incidents from popular 
television shows, movies, cartoons, 
and even commercials were often 
interwoven in the students' 
writing. 
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Writte?i-Response Study 
Grade 6 Social Studies 
Design 

Students' writing performance has 
been of particular interest to 
educators and to the public since 
the beginning of the Achievement 
Testing Program. As a consequence 
of this interest in how well 
students write, a study was 
initiated to compare 1993 writing 
with 1989 writing and 1985 
writing in Grade 6 Social Studies. 
This was a descriptive study that 
required the teacher-readers to 
take a research or reader- as- 
observer stance toward the papers 
they read. Teacher-readers 
described features of the 1993, 
1989, and 19*85 writing in two 
scoring categories: 

State and Support an Opinion/ 
Position 

•Evidence of position 
•Reasons/arguments used to 
support the position taken 
•Quality of examples to support 
the position taken 



Quality of Language and 
Expression 

•Organization 
-focus 

-coherent order 
-connections between main 
ideas and/or detail 

•Vocabulary and Conventions 
-effective words and 

expressions 
—correct grammar, punctuation, 

spelling, capitalization, 

sentence structure 
-effective communication 

In 1985 and 1989, the written- 
response assignment involved short 
answers and one extended response. 
In 1993, the assignment was limited 
to one extended response. In order 
to facilitate comparison of similar 
assignments, only the extended 
response from 1985 and 1989 was 
compared with the 1993 
assignment. 

Comparisons were made at two 
standards: Acceptable (3), which 
represents work at an acceptable 



level for students completing 
Grade 6, and Excellent (5), which 
represents outstanding work for 
students completing Grade 6. 

Methodology 

Papers read in the study were 
selected at random from papers 
that received scores of Acceptable 
(3) or Excellent (5) on the June 
1993, June 1989, and June 1985 
achievement tests. 

A group of ten experienced teacher- 
readers, representing all major 
regions of the province and a 
variety of school settings, reviewed 
the selected papers. 

Working alone and in pairs, and 
then discussing papers as a group, 
the teacher-readers described the 
papers for features of state and 
support an opinion/position and 
quality of language and expression. 

They then compared their 
descriptions of 1993, 1989, and 
1985 Acceptable (3) and Excellent 
(5) papers to draw conclusions. 
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Results 



Acceptable Standard (3) 

Teacher-readers felt that the 
papers judged to achieve the 
Acceptable Standard (3) were 
slightly better in 1993 than in 1989 
and 1985. Teacher-reader 
comments of these papers are 
summarized below. 



Acceptable Standard (3) in Grade 6 Social Studies 



Key Features of 1993 Papers 


Key Features of 1989 Papers 


Key Features of 1985 Papers 


State and Support an Opinion/Position 


Persuasiveness and Logic of Supporting 


Persuasiveness and Logic of Supporting 




Arguments 


Arguments 


•identifiable position presented 


•identifiable position presented 


•identifiable position presented 






•mechanical argumentation 


•ideas presented but not developed 


•ideas presented but not developed 


•ideas presented but not developed 


•use of implicit rather than explicit ideas 


•use of implicit rather than explicit ideas 


•use of implicit rather than explicit ideas 


•limited defence of position, supported 


•limited defence of position, supported 


•limited defence of position, supported 


by generalities 


by generalities 


by generalities (not always accurate) 


•position supported by personal experiences 


•position supported by personal experiences 


•position supported by personal experience 






(sometimes presented as fact) 


•some opinions reflect extremes found in 






media today 






•limited development, use, and 


•limited development, use, and 


•limited development, use, and 


understanding of social studies concepts 


understanding of social studies concepts 


understanding of social studies concepts 


•limited understanding of levels of 






government 






Quality of Language and Expression 


| Quality of Language and Expression 


Quality of Language and Expression 


•paragraphing skills evident, use of 


•poor organization and structure 


•poor organization and structure 


closing paragraph 






•frequent errors in grammar, punctuation, 


•frequent errors in grammar, punctuation, 


•frequent errors in grammar, punctuation, 


spelling, and sentence structure 


spelling, and sentence structure 


spelling, and sentence structure 


•language general not specific 


•language general not specific 


•language general not specific 


•specific words attempted 


•limited vocabulary 


•limited vocabulary 
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Excellent Standard (5) 

Teacher-readers felt that papers 
judged to achieve the Excellent 
Standard of Excellence (5) were 
superior to papers from 1989 and 



1985. A summary of the teacher- 
reader comments on differences 
and similarities in the two 
reporting categories follows. 



Excellent Standard (5) in Grade 6 Sociai Studies 



Key Features of 19S3 Papers 


Key Features of 1989 Papers 


Key Features of 1985 Papers 


Slate and Support an Opinion/Position 


Persuasiveness and Logic of Supporting 


Persuasiveness and logic of Supporting 




| Arguments 


Arguments 


•strong, clear position is taken 


•clearly identifiable position taken 


identifiable position presented 


•position supported by one or more ideas 


•position supported by one or more relevant 


•position supported by one or more 




ideas 


arguments (personal experiences) 


•convincing arguments well developed 


•sound arguments developed 


•arguments not fully developed 


•consistent position throughout 


•arguments lack consistency 


•arguments lack consistency 


•logical sequence to arguments that are 


•supportive examples are specific and 


•examples very specific 


based on cause/effect relationships 


relevant 




•solid understanding of sociai studies 


•social studies concepts used 


•limited inclusion of social studies concepts 



concepts demonstrated throughout 

•good understanding of levels of 
government 



Quaiity of Language and Expression 



•writing fluent and we organized 
paragraphs containing a main idea and 
supporting facts 

•few minor errors in conventions 



Quality of Language and Expression 



Quality of Language and Expression 



•writing generally fluent and good paragraph 
development 

•some minor errors in grammar, punctuation, 
spelling, and sentence structure 



Hcgica! organization but weak or missing 
conclusion 



•limited number of errors in grammar, 
punctuation, spelling, and sentence 
structure 



•vocabulary is accurate and effective 



-language is appropriate and specific 



•language is general not specific 
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Concluding Comments 



The question that guided this study 
was: Has writing by Grade 6 
Social Studies students 
improved from 1985 to 1993? 

Teachers participating in the study 
thought that, in 1993, students 
achieving the acceptable standard 
(3) produced writing of similar 
quality when compared to their 
1985 and 1989 counterparts. They 
thought that 1993 students 
achieving the excellent standard 
produced higher calibre writing 
when compared to their 1985 and 
1989 counterparts. 

A major change was made to the 
writing component of the 1993 
achievement test. In 1993, students 
addressed only one written- 
response question, whereas in 1935 
students answered four short 
written-response questions before 
attempting the extended response, 
and in 1989 students answered six 
short written-response questions 
before attempting the extended 
response. In addition, students in 
1993 had more time to answer the 
extended-response question. 
Within the confines of a test 
situation, some students in 1985 
and 1989 may have experienced 
oversaturation with the test as well 
as lack of time before attempting 
the extended response. 
Teacher-readers made the 



following observations related to 
the changing nature of the writing 
over the three test years. 

The teacher-readers felt that the 
1993 issue was more general and 
abstract than the 1985 and 1989 
issues for the written-response 
question. They felt that while the 
additional time students had to do 
the writing assignment produced 
better writing than in 1985 and 
1989, a more specific and concrete 
question might have provided 
students a better opportunity to 
achieve higher levels. 

The teacher-readers thought that 
the different prompts used on the 
three tests may have influenced 
the type of writing produced by 
students. In 1985 and 1989, the 
short-answer questions were 
prompts in themselves. The 1993 
prompt was more detailed than the 
others and included visuals. 
However, the teacher-readers 
indicated that the writing prompt 
may have been misleading and 
confusing for students not 
achieving provincial standards. It 
may have been difficult for some 
students to relate the information 
provided in the prompt to the 
assignment, and it may have 
served as a distraction rather than 
as a help to students in their 



writing. The teacher-readers felt 
that this may be true especially for 
students marginally achieving at 
the acceptable standard or lower. 
They felt that higher acHeving 
students were better able to use 
the prompt. 

In addition, the teacher-readers 
thought that the 1993 assignment 
limited student choice, as the 
information stated that "there are 
two positions to this issue" and did 
not state that students could take 
a qualified position. However, this 
did not prevent a number of 
students from taking qualified 
positions in their writing. 

Teachers also indicated that, in 
some cases, students' planning did 
not relate to their writing. Once 
students began writing, they t : ther 
forgot about the plan or found 
going back and forth to the 
planning pages inconvenient. This 
is common practice for students. 
The teacher-readers suggested that 
a tear-out sheet should be provided 
for the written response so that 
students could make better use of 
planning. 
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Section 9 
Age 9 Achievement 



Methods of improving the current 
assessment program have been 
explored over the last few years. 
One of the methods under 
investigation has been to assess a 
particular age group of students 
rather than particular grade levels. 
In 1993, in >rder to collect age- 
based information about student 
achievement in Language 
Learning, a special study to sample 
nine-year-olds in grades 2, 3, and 4 
classrooms was undertaken. 

Key questions that guided the 
design of this study included: 

• What level of performance do 
nine-year-old students 
demonstrate in reading and 
writing? 

♦ How do levels of performance 
compare between students of the 
same age but different grades in 
school? 

* What are some of the 
administrative concerns related 
to age-based assessment? 

• Do teachers support age-based 
assessment? 

Design 

Both the reading and writing 
components of the Age 9 
assessment were the same as the 
Grade Level 3 Language Learning 
assessment. 



The written component was 
marked using a levels-based 
scoring guide. The reporting 
categories (Content and 
Development, Use of Language, 
and Conventions) were the same as 
those used for the regular Grade 
Level 3 assessment. However, the 
five scales were as follows: 

• May Be Beyond Grade Level 4 
9 Grade Level 4 

• Grade Level 3 

• Grade Level 2 

• Not Yet At Grade Level 2 

The specific criteria outlined in 
each level were directly related to 
the descriptors found at the 
various levels in the Language 
Learning component. A sample of 
papers from students aged 9 and in 
Grade Level 3 were included in the 
design. The papers were marked a 
second time for this study using 
the levels-based scoring guide. 

Methodology 

A total of 1 032 stuaents from 29 
schools participated in the Age 9 
study. Schools were chosen 
randomly and reflected a 
provincially representative sample. 
The number of students from 
grades 2, 3, and 4 classrooms in 
the Age 9 sample was in the same 



proportion as the number of Age 9 
students in each grade for the 
province overall. 

School principals were contacted by 
phone to confirm administrative 
procedures and to collect an accurate 
count of students in their school who 
would be nine years of age and not in 
Grade Level 3 on the date the 
assessment was administered. 

Both the reading and writing 
components of the Age 9 assessment 
were administered at the same time 
and followed the same 
administration procedures as the 
Grade Level 3 Language Learning 
assessment. 

Results 

The findings of this study are 
discussed in conjunction with the 
key questions that guided the project 
design: 

• What level of performance do 
nine-year-old students 
demonstrate in reading and 
writing? 

The students' performance in 
reading and writing was scored 
according to level of performance. 
Provincial results are shown in 
tables 9-1, 9-2, and 9-3. 
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Table 9-1 

Age 9 Study, Language Learning 

Percentage of Students Achieving Level of Performance on Both Components of the Test 

June 1993 



Level of Performance* 


Reading Component 


Writing Component 


May Be Beyond Grade Level 4 


18.7 


5.9 


Grade Level 4 


25.6 


20.0 


Grade Level 3 


31.6 


35.9 


Grade level 2 


16.5 


29.0 


Not Yet At Grade Level 2 


7.6 


9.1 



*Level of performance in this context refers to one of the seven grade levels outlined in the Language Learning component of the 
Program of Studies (revised 1993). 



The highest percentage of students 
in the Age 9 sample demonstrated 
performance at Grade Level 3, 



with fewer students demonstrating 
performance at Grade Level 2 and 
Grade Level 4. Overall, 



performance level of Age 9 
students was higher in reading 
than in writing. 



Table 9-2 

Age 9 Study, Language Learning 

Percentage of Students Achieving Level of Performance by Grade 
Reading Component 

June 1993 



Grade 


Number of 
Students 


Not Yet at 
Grade Level 2 


Grade 
Level 2 


Grade 
Level 3 


Grade 
Level 4 


May Be 
Beyond Grade 
Level 4 


Total 


2 


13 


61.5 
'10.1 


23.1 


15.4 


0.0 


0.0 


100.0 


3 


495 


23.3 


31.9 


21.6 


13.1 


100.0 


4 


524 


3.8 


10.1 


31.7 


30.0 


24.4 


100.0 


TOTAL 


956 


7.1 


15.4 


30.9 


26.8 


19.6 


100.0 



Table 9-3 

Age 9 Study, Language Learning 

Percentage of Students Achieving Level of Performance by Grade 
Writing Component 

June 1993 



Grade 


Number of 
Students 


Not Yet at 
Grade Level 2 


Grade 
Level 2 


Grade 
Level 3 


Grade 
Level 4 


May Be 
Beyond Grade 
Level 4 


2 


13 


69.2 


30.8 


0.0 


0.0 


0.0 


3 


495 


12.1 


32.3 


36.8 


15.6 


3.2 


4 


524 


4.8 


25.8 


36.0 


24.8 


8.6 


TOTAL 


956 


8.3 


29.0 


35.6 


20.8 


6.1 



Total _ I 

100.0 j 
100.0 | 
100.0 ! 
100.0 ! 
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• How do levels of performance 
compare between students of 
the same age but different 
grades in school? 

Tables 9-2 and 9-3 indicate that 
student performance does appear to 
be related somewhat to grade level; 
that is, nine-year-olds in Grade 
Level 4 demonstrated higher levels 
of reading and writing than nine- 
year-olds in Grade Level 3. 
Similarly, nine-year-olds in grade 
levels 3 and 4 performed at higher 
levels than nine-year-olds in Grade 
Lev A 2. However, a high percentage 
of students still did not achieve an 
acceptable level of performance for 
the grade they were in. Of students 
in Grade 2 classrooms, 
approximately 62% did not achieve 
at Grade Level 2 in reading and 
69% did not achieve at Grade Level 
2 in writing. Of Grade 3 students, 
33% did not achieve at grade level 
in reading and approximately 44% 
did not achieve at grade level in 
writing. Approximately 45% of the 
students in Grade 4 classrooms did 
not perform at grade level in 
reading; 66% of them achieved 
below grade level in writing. 



It is unclear whether the 
differences among grades are due 
to promotion policies and the type 
of instruction provided for students 
who are retained, to the number of 
years students have been in school, 
to grade-specific curricula being 
taught, or to other factors. 

• What are some of the 
administrative concerns 
related to age-based 
assessment? 

Findings from the administration 
of the Age 9 assessment showed 
few reactions to the concept of age- 
based assessment. Only one 
principal expressed a concern 
about administering the 
assessment to only some students 
from classes organized by grade. 

• Do teachers support age- 
based assessment? 

The 150 teachers who marked the 
Grade Level 3 compositions were 
asked if they would support 
provincial assessments for nine- 
year-olds instead of for Grade 
Level 3 students. Only 23% of 



those teachers were in favour of a 
move toward age-based 
assessment. Most expressed 
concerns about pulling a sample of 
students away from their regular 
classes and teachers. 

Teachers who marked the Age 9 
writing component were generally 
positive about the levels-based 
scoring guide that was used. 



Concluding Comments 

Through this study, we learned 
that it is possible to conduct age- 
based assessment. However, the 
approach does not correspond to 
the administrative structure that 
currently exists in schools. 
Administration of age-based 
assessment, given the nature and 
purpose of the provincial 
Achievement Testing Program, can 
be disruptive, and the assessment 
does not provide teachers, 
administrators, and parents with 
information that is most useful to 
them. These -findings will 
contribute to the planning and 
structuring of future assessments. 
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Appendix A 


1 

V 


Standards ' 



The move toward a results-based 
curriculum has re-emphasized the 
need for a clear delineation of 
standards and their purpose. 
All standards and all methods 
of setting standards require 
judgment. 

The process of setting a 
standard can be only as good as 
the judgments that go into it. 
The standard will depend on 
whose judgments are involved 
in the process. In this sense, all 
standards are subjective. Yet, 
once a standard has been set, 
the decisions based on it can be 
made objectively. Instead of a 
separate set of judgments for 
each test-taker, you will have 
the same set of judgments 
applied to all test-takers. 
Standards cannot be objectively 
determined, but they can be 
objectively applied. 1 

Definitions 

The Achievement Testing Program 
is directly concerned with three 
different but related standards. 
These provincial standards are 
curriculum standards, achievement 
standards, and assessment 
standards. 

* Curriculum Standards are 

the expected student learnings 
sequenced into grade levels. 
They include broad statements 
of knowledge, skill, and attitude 
expectations against which 
student performance is judged. 
These standards are established 



in the process of curriculum 
development and are found in 
the Program of Studies 
document produced for each 
course. 

• Achievement Standards are 

judgments that specify what 
percentages of students are 
expected to achieve an 
acceptable and an excellent level 
of performance in relation to 
each course of studies, i.e., the 
relevant curriculum standards. 
It is important to point out that 
this judgment is not a prediction 
of the percentage of students 
who will actually achieve 
acceptable or excellent levels of 
performance but rather is a 
specification of the percentage of 
students at a given age or grade 
in school who are expected to 
achieve an acceptable or 
excellent level. These standards 
apply to school, jurisdiction, and 
provincial performance. 

• Assessment Standards are the 
criteria adopted forjudging 
actual student achievement 
relative to curriculum standards. 
They are ultimately expressed 
and applied as test scores. They 
are derived from answers to 
questions such as: 

What scores must a student 
obtain or how many 
questions on a given test 
must a student answer 
correctly in order for his/her 



performance on the test to be 
judged as acceptable or 
excellent? 

These standards apply to 
individual student performance. 

Indirectly, the Achievement 
Testing Program influences local 
targets, 

• Local Targets are the objectives 
set in schools and jurisdictions to 
assist students in moving toward 
or exceeding the provincial 
assessment standards. These 
local targets reflect the specific 
needs of individuals and groups 
within a specific community. 

The Student Evaluation Branch 
is responsible for establishing 
and reporting the provincial 
assessment standards and shares 
responsibility with the Curriculum 
Branch for establishing provincial 
achievement standards as they 
relate to the Achievement Testing 
Program. 



l Passi?ig Scorrs; Samuel A. Livingston, Michael J. Zieky; Educational Testing Service, 1982. 
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Purpose 

The purpose of defining standards, 
and the subsequent process of 
setting the provincial assessment 
standard within the sphere of the 
Achievement Testing Program, is 
•to answer questions such as: 

• What is acceptable and 
excellent performance in 
relation to the curricular 
expectations for students at 
the age or grade being tested? 

• What percentages of students 
at the age or grade being tested 
ought to achieve an acceptable 
and excellent level, assuming 
adequate teaching and 
resources? 

• What scores on a specific test 
shall reasonably represent 
acceptable and excellent 
performance respectively? 

• What are the prevailing 
strengths and weaknesses of 
Alberta students in relation to 
the curriculum being tested? 



In essence, for each test and level 
of performance, the assessment 
standards setters are challenged 
to answer the question: 

What score must a student 
obtain or how many questions 
must a student answer 
correctly to be judged as 
having achieved an acceptable 
or excellent standard? 

Satisfactory performance is 
demonstrated when the percentage 
of students scoring at or abo*~e the 
assessment standards is equal to 
or greater than the achievement 
standards. 

In this report, tables in sections 2, 
3, 4, and 9 dealing with standards 
show what percentage of students 
achieved standards on each major 
component and on the total test. 



The Assessment 

Standard-Setting 

Process 

Figure A-l shows the model 
established in 1991 by the Student 
Evaluation Branch to set 
standards for achievement tests. 
The objective was to widen the 
process of setting assessment 
standards as much as possible 
over previous years and 
especially to provide for 
community input and 
feedback* The process involves 
individuals and groups making 
judgments that contribute to 
establishing the assessment and 
achievement standards. 

Except for the members of the 
Public Advisory Committee, 
the members of all committees 
outlined in Figure A-l are expected 
to be highly knowledgeable about 
both the curriculum and the 
learning characteristics of the 
students who are writing the tests. 
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Figure A- 1 

Process Model for Standard Setting 
Achievement Testing Program 

June 1993 

INPUT 



/ \ 




PRODUCT 

* 

ASSESSMENT STANDARDS 
Raw scores (cut-off scores) representing 
Acceptable Standard and 
Standard of Excellence 
and 

ACHIEVEMENT STANDARDS 
Percentage of students expected to 
achieve or exceed Assessment Standards 



1. Curriculum and Test 
Developers Committee 

There is one Curriculum and 
Test Developers Committee for 
each achievement test. The 
committee consists of Alberta 
Education consultants, 
curriculum developers, and 
test development specialists. 
Their objective is to recommend 
assessment standards to the 
Final Standards Review 
Committee. They may also 
review and make 
recommendations on 
Achievement Standards. 



2* Test Review Committee 

There is one Test Review 
Committee for each achievement 
test. Experienced and 
knowledgeable educators — 
teachers, superintendents, 
university professors, and 
Alberta Education consultants — 
meet with the developers of 
each test. Their purpose is to 
review the test and make 
recommendations for 
improvements where necessary. 
In addition, the committee 
reviews the appropriateness of 
current courae achievement and 
assessment standards. 



3. Public Advisory Committee 

A Public Advisory Committee, 
composed of representatives from 
the general public, met again in 
1993 and discussed standards. 
See Appendix B for more 
information on the proceedings 
of this committee. 

4. Teacher Standards 
Committee 

Approximately 20 experienced 
teachers from different areas of 
the province are selected to sit on 
a Teacher Standards Committee 
for each test. To be selected for a 
committee, a teacher must have 
been teaching in the grade and 
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subject area for the previous two 
years. At present, the procedures 
used to aid teachers in setting 
the assessment standards are 
the modified Angoff method for 
Mathematics and Science 
subjects and the Nedelsky 
method for Humanities subjects. 2 
The teachers make judgments 
about appropriate assessment 
standards and recommendations 
to the Final Standards Review 
Committee. As well, they review 
Achievement Standards for 
appropriateness. 

5. Analytic Services Committee 

The Analytic Services Unit of the 
Student Evaluation Branch is 
the professional quality control 
and advisory group for the 
complete process of standard 
setting. There is a committee of 
professionals in psychometrics 



And statistics for each test. A 
critical function is to ensure that 
Alberta Education's standard- 
setting procedures produce 
technically valid results that 
meet the strict requirements of 
high quality professional studies. 
The committee makes 
recommendations for assessment 
standard-settingprocedures, 
leads individual group 
discussions, and recommends 
improvements where necessary. 
Another function is to determine, 
independently, assessment 
standards through statistical 
analyses of current student 
achievement data. 

6.The Final Standards Review 
Committee 

The Final Standards Review 
Committee consists of 
representatives from the above 



committees and is chaired by the 
Director of Student Evaluation. 
In separate sessions, the 
recommendations of all test 
committees are presented and, 
through consensus, final 
Assessment Standards are 
adopted for each test. 

1993 Assessment and 
Achievement Standards 

Tables A-l to A-3 show the 
assessment and achievement 
standards adopted in June 1993 
by the Final Standards Review 
Committee for the grades 3, 6, 
and 9 achievement tests. The 
tables also show the percentages of 
students achieving provincial 
assessment standards. 



2 "A Consumers Guide to Setting Performance Standards on Criterion-Referenced Tests," Ronald A. Berk: Review of Educational 
Research, Spring, 1986, Volume 56. 



Table A 1 

Grade Level 3 English Language Learning 
Assessment and Achievement Standards 

June 1993 



Category 

Total Test 
Writing Skills 
Reading Skills 



Category 

Total Test 
Writing Skills 
Reading Skills 



Provincial 
Assessment* 

Standard 
(Raw Score) 

79/100 
15/20 
33/40 



Provincial 
Assessment* 

Standard 
(Raw Score) 

48/100 
9/20 
19/40 



Standard of Excellence 



Percentage Achieving 
Assessment Standard 



14. i 
14.5 
28.8 

Acceptable Standard 



Percentage Achieving 
Assessment Standard 



82.2 
81.3 
82.2 



Provincial 
Achievement* 
Standard 

(%) 

15 
15 
15 



Provincial 
Achievement* 
Standard 

(%) 

85 
85 
85 



ERLC 



*The Provincial Assessment Standard is a score determined by appropriate standard-sotting procedures and is the lowest score a 
student must achieve for his/her performance to be judged acceptable or excellent in relation to curricular expectations. 

**The Provincial Achievement Standard refers to the percentage of students expected to meet the Provincial Assessment Standard. 
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Table A 2 

Grade 6 Social Studies 

Assessment and Achievement Standards 

June 1993 



Category 


Standard of Excellence j 


r i 
Provincial 

Assessment 

Standard 

(Raw Score) 


Percentage Achieving 
Assessment Standard 


Provincial 
Achievement j 
Standard ; 

(%) ! 


Total Test j 80/100 
Skills j 50/64 
| Knowledge 20/24 


8.0 
6.2 
28.7 


15 | 

15 

15 




i 

I 
i 

Category 


Acceptable Standard j 


Provincial 
Assessment 

Standard 
(Raw Score) 


Percentage Achieving 
Assessment Standard 


Provincial j 
Achievement 
Standard 

(%) 


Total Test 
j Skills 
| Knowledge 


49/100 
30/64 
13/24 


80.8 
77.2 
83.5 


85 
85 
85 



Table A 3 
Grade 9 Science 

Assessment and Achievement Standards 

June 1993 







Standard of Excellence 








Provincial 




Provincial | 




Assessment 


Percentage Achieving 


Achievement 


Category 


Standard 


Assessment Standard 


Standard 




(Raw Score) 




(%) \ 


Total Test 


52/65 


h , 

17.1 


15 ! 


Skills 


23/29 


21.5 


15 ! 


Concepts 


j 29/36 


! 19.3 
! 


15 I 



Category 

Total Test 

Skills 

Concepts 



Provincial 
Assessment 

Standard 
(Raw Score) 

33/65 
15/29 
18/36 



Acceptable Standard 



Percentage Achieving j 
Assessment Standard I 



77.6 
74.8 
80.3 



Provincial 
Achievement 
Standard 

(%) 

85 
85 
85 
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Appendix B 

Public Itcrieiv of 
Standards and Results 

v 



On July 21 and 22, 1993, 
representatives of seventeen 
business, professional, and 
community organizations reviewed 
achievement test results and 
discussed standards. This meeting 
was planned to ensure that input 
from groups representing "public" 
interests would be considered in the 
overall reporting of results. The 
purpose of the meeting was to 
gather direct public input into the 
question: 

How well should Alberta 
students achieve in Grade Level 3 
Language Learning, Grade 6 
Social Studies, and Grade 9 
Science? 

The two-day meeting was designed 
to collect information that would 
contribute to a better understanding 
of how well Alberta students are 
doing. This was accomplished by 
exploring a_id discussing answers to 
four questions. Each question is 
listed below, along with the results 
of the discussions. 

Question: 

What do members of the public 
expect students to know and be 
able to do in Grade Level 3 
Language Learning, Grade 6 
Social Studies, and Grade 9 
Science? 

For each subject area, participants 
listed the knowledge and skills that 
they felt were important learnings 
for students to achieve at each 
grade level. 



The following learnings were 
highlighted by the group as 
important: 

Language Learning 3 

Students should: 

• know how to organize their 
writing using a prompt 

• know how to work cooperatively 

• know the correct use of sentence 
structure, spelling, 
paragraphing, grammar 

• be able to synthesize/draw 
conclusions/make generalizations 

• be able to interpret what they 
read/see 

• be able to follow directions 

• be able to determine general 
word meanings and main ideas 

• be able to relate experience/ 
imagination to the written word 
using appropriate vocabulary. 

Social Studies 6 
Students should: 

• know how government works 

• know the difference between 
needs and wants 

• know how to debate 

• be able to articulate a position 
and present supporting 
arguments 

• be able to organize thoughts 



Science 9 
Students should: 

• know the scientific method 

• know general science concepts 
(facts and relationships) 

• know how science impacts on 
society 

• be able to manipulate formulas 
and variables 

• be able to use process skills 

• be able to apply their 
mathematics background 

• be able to demonstrate scientific 
literacy 

Question: 

How closely do the public 
expectations about student 
learning relate to provincial 
standards? 

Participants reviewed the 1993 
Language Learning, Social 
Studies, and Science tests. There 
was essentially a "match* between 
the public expectations of what 
students should know and be able 
to do and the knowledge and skills 
neoded to answer the questions in 
each of the tests. 

Participants were pleased with the 
variety of testing methods 
(numerical response, visuals, 
performance-based assessment) 
used to provide information about 
student achievement. 
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Participants expressed concern 
that the prompt for the Social 
Studies 6 written component may 
have provided students with 
extraneous information and 
therefore confusing rather than 
clarifying the assignment for them. 

Question: 

How well did students actually 
perform on the achievement 
tests? 

Participants identified questions in 
the test that corresponded to 
knowledge and skills they felt were 
most important for students to 
learn. In addition, they discussed 
the percentage bf students in 
Alberta that should answer each 
question correctly to meet 
provincial standards. Results 
showing how well students 
actually performed on these 
questions were also presented to 
the group. As well, an initial 
analysis of the coxiiposition 
component of the Language 
Learning and Social Studies test 
was presented. 

In each area, students achieved 
some of the expectations 
participants set for specific 
learnings. For other learning. s 
achievement was below 
expectations. These areas are 
listed below. 

Language Learning 3 

Met Standards: 

• be able to use background 
knowledge/experience 

• understand main ideas, key 
details 

• be able to express ideas in a 
coherent written form 



108 



• be able to draw conclusions, 
make generalizations. 

Below Standards: 

• be able to recognize and 
interpret word meanings 

• be able to make judgements/ 
interpretations 

Social Studies 6 

Met Standards: 

• know basic concepts 

• be able to identify geographical 
relationships 

• be able to synthesize/interpret 
information 

Below Standards: 

• be able to differentiate between 
facts and opinion 

• be able to evaluate information 

• be able to apply knowledge in a 
global sense 

• be able to take and support a 
clear position in written form. 

Science 9 
Met Standards: 

• know scientific processes (fluid 
systems, buoyancy, gravity) 

• be able to interpret graphs and 
diagrams 

• be able to analyze and evaluate 
given information 

Below Standards: 

• know scientific terms 

• be able to apply knowledge to 
new situations 
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Question: 

Did the group feel that 
students' performance on the 
achievement tests was good 
enough? 

Language Learning 3 

Generally, participants felt that 
actual student performance met 
what they felt was an acceptable 
standard. 

They did feel, however, that the 
students in the category, 
"approaches the acceptable 
standard" should not be forgotten 
about. These are the students we 
need to focus on next to bring them 
up to an acceptable level. Our 
ultimate goal is to have 100% of 
our students achieve the 
acceptable level. 

Social Studies 6 

Most participants did not feel that 
students' performance is good 
enough. 

The biggest problem seemed to be 
that standards are too low. Also, 
there are not enough students 
achieving the acceptable standard 
or the standard of excellence. 

Science 9 

Committee members were almost 
evenly split between "yes" an d "no" 
responses to the questions of 
whether student performance was 
good enough. 

Students generally met the 
committee's expectations, but 
participants acknowledged that 
there was still room for 
improvement. As well, there were 



differences in personal views 
regarding the level of scientific 
knowledge being assessed at 
Grade 9. 

Other Comments from 
Participants 

Several other comments were 
made. All members of the 
committee had an opportunity to 
provide feedback regarding the 
organization and activities of the 
two-day meeting. Their comments 
are summarized below: 

• All participants supported the 
use of a variety of testing 
methods, such as numerical- 
response questions, visuals, and 
performance-based assessment, 
in allowing a greater degree of 
success for more students. 



• Many participants suggested 
that testing should begin at a 
relatively early age so that there 
is time to respond to any 
problems that may show up. 

• Participants felt that test scores 
are affected by a variety of 
factors, including attitude 
(motivation) and curriculum 
demands. 

• Many participants agreed that 
the Student Evaluation Branch 
needs to continue to address test 
bias, especially in relation to 
Native culture. 

• Participants indicated that this 
type of "public meeting" activity 
should continue, possibly on a 
broader scale. 



A 1? mal Note 

The views of participants 
regarding the achievement of 
Alberta students were used to help 
set the assessment standards for 
the 1993 achievement tests. 

A more detailed report on the 1993 
meeting was distributed to 
participants and is available upon 
request. Please contact Yvonne 
Johnson at 427-0010 to obtain a 
copy of this report. 

The next meeting for public review 
of achievement standards will be in 
July 1994. 
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Appendix C 

* . (Guidelines for 
Interpreting Results 



Jurisdiction and 
School Reports 

The jurisdiction and school reports 
describe the results achieved by 
students who wrote achievement 
tests in Grade Level 3 Language 
Learning, Grade 6 Social Studies, 
and Grade 9 Science in June 1993. 
These reports include provincial 
results and allow school board 
members, superintendents, 
principals, and teachers to 
compare jurisdiction and school 
results with provincial standards 
and levels of achievement. 

To decide if the achievement of 
their students is "good enough," 
educators at the school and 
jurisdiction levels can compare 
their results to: 

• provincial standards 

• local targets for student 
achievement 

• the achievement of students 
throughout the province of 
Alberta 

• school and jurisdiction results 
from the last provincial 
administration of tests in these 
subjects in 1989 

Use of the Juirisdiction 
and School Reports 

The Achievement Testing Program 
is designed primarily to provide 
group results. However, individual 
student results are provided to 
schools and may be used in 
conjunction with other information 
to plan individual programs for 
stude*. 3. Also, parents are entitled 



to know how their child did on the 
test. As required by regulation, 
each student's results must be 
included in the student's school 
record file. 

The test results are not intended to 
be used as a basis for: 

• making decisions about student 
placement or promotion 

• evaluating teacher performance 

• comparing performance between 
or among schools 

The interpretation of results from 
the achievement tests involves 
considering the many factors that 
contribute to achievement. 
Although the tests are designed to 
assess provincial standards as 
reflected in the general learner 
expectations of the Program of 
Studies, many important aspects of 
learning cannot be measured by 
time-limited, paper-and-pencil 
tests. 



Standards 

Achievement tests and the school 
and jurisdiction reports are 
designed to provide information 
about how well students are doing 
compared to established standards. 
Two standards are set for each 
achievement test. The Provincial 
Assessment Standard is the lowest 
score a student must achieve for 
his or her test performance to be 
judged "acceptable" or "excellent" 
in relation to curricular 
expectations. Stardards for 
"excellent" and "acceptable" were 



set for the total test and for the 
two major components of each test. 

Provincial Achievement Standards 
specify the percentage of students 
expected to meet the assessment 
standards. 

Table 2 in the school and 
jurisdiction reports shows the 
assessment and achievement 
standards for the two components 
of each test and for the total test, 
together with the number of 
students who met the standards. 
The table also shows whether the 
number of students in that school 
or jurisdiction who have achieved 
the standards is significantly 
different from the expected 
number. A difference is reported 
as significant when there is a 5% 
or smaller probability that a 
difference of that size could occur 
by chance. For schools and 
jurisdictions with fewer than five 
students, significance was not 
calculated. 

Although the statistical tests take 
the number of students into 
consideration, results for groups of 
fewer than 25 students must be 
interpreted with particular 
caution. 

Achievement-Over-Time- 
Study 

Some schools participated in a 
study of changes in achievement 
over time. Students in tiiese 
schools were randomly assigned to 
one of three alternative forms of 
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the achievement test for their 
gTade. Two forms included items 
from the tests used in the same 
subjects in 1985 and 1989, as well 
as some items from the 1993 test. 
The third form was identical to the 
1993 test. A full explanation of the 
design and the results of the study 
are included in this Provincial 
Report. 

To treat all students in an 
equitable manner, scores on the 
alternative forms were equated so 
that students receive a mark 
equivalent to one they would have 
received if they had written the 
1993 test. This means that the 
results for all students are 
included in most tables. 

For those schools and jurisdictions 
that participated in the study, 
multiple-choice results by 
reporting category and individual 
student subscale scores are 
presented in separate tables for 
each form. 

Also, tables that refer to frequency 
distributions for individual 
multiple-choice and numerical- 
response questions are different for 
schools participating in the study. 
For non-participating schools, 
these tables include distributions 
for the 1993 items only. Provincial 
results in these tables for 1993 
items differ slightly between the 
reports for schools in the study and 
those not in the study. This is 
because the provincial results for 
the non-participating schools are 
based only on the responses of 



students writing the 1993 test, 
whereas for participating schools 
results for 1993 items appearing on 
the 1985 and 1989 forms of the test 
are also used. For schools 
participating in the achievement- 
over-time study, distributions are 
provided for items that appear on 
all forms. 

Results for Individual 

Multiple-Choice 

Questions 

The responses to each item may 
total less than 100% because some 
students did not answer some of 
the questions. 

Grade Level 3 
Language Learning 

The written-response section of the 
Grade 3 Language Learning 
Achievement Test was scored on 
three scales. To calculate the total 
score on the written-response 
section, a student's score on the 
first scale (Content and 
Development) was multiplied by 
two and added to the scores 
achieved on the other two scales 
(Use of Language and , 
Conventions). Thus, the totaK 
possible score for written-response 
is 20. The reading and writing 
components of the test were g*yen 
equal weighting in calculating the 
total test score. To calculate a 
student's total score, the written- 
response score was multiplied by 
50/20, the reading score was 
multiplied by 50/40, and the two 
scores were added to give a total 
score out of 100. 



Grade 6 Social Studies 

Analysis of the student responses 
to multiple- choice items on the 
Grade 6 Social Studies 
Achievement Test revealed 
problems with five items. Three 
items were deleted from the test 
(22, 33, and 45), and two items (9 
and 49) had two correct answers in 
the possible responses and were 
marked right for both answers. 
None of these items was used for 
the achievement-over-time study; 
thus, the regular form and Form F 
had 47 multiple-choice items and 
forms D and E had 50 multiple- 
choice items. 

All forms of the test' had the same 
written-response assignment. This 
section was scored on two scales. 
To calculate a total score for the 
written-response section, the score 
awarded on the first scale (State 
and Support an Opinion/Position) 
is multiplied by two and added to 
the score awarded on the second 
scale (Quality of Language and 
Expression). The maximum 
possible score is 15 for written 
response. 

The two major reporting categories 
for the test are knowledge and 
skills. The knowledge score is 
based on multiple-choice items 
only. The skills score is a weighted 
combination of multiple-choice and 
written-( >mponent scores. 

Total test scores were calculated 
according to the formula indicated 
in the following table. 



I 



Test Form 

Regular and Form F 
Form E 
Form D 



Number of Multiple-Choice Multiple | Written 

Questions Choice | Response 

(70% of Total) W ghting , (30% of 

Knowledge j Skills j Factor | total) 

24 23 70/47 15 
20 30 j 70/50 i 15 

25 25 I 70/50 j 15 



Total 

= (multiple-choice score) x 
weighting factor + 2 x 
wtitten response 

100 
100 
100 
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Because the knowledge component 
consists entirely of scores on 
multiple-choice items, the 
knowledge scores are reported as 
unweighted raw scores. When 
scores on forms E and D are 
equated to the regular form, the 
equated knowledge scores have the 
same maximum score as the 
regular form; that is, 24. 

The skills component includes both 
multiple-choice and written- 
response questions, and is reported 
only as a weighted combination. 
The maximum score on the skills 
component for the regular form 
and Form F is 64. For Form E, the 
maximum score is 72; for Form D, 
it is 65. When scores on forms E 
and D are equated to the regular 
form, the equated skills scores 
have the same maximum score as 
the regular form; that is, 64. 



Comparing Results to 
Average Scores 

To determine if students are "doing 
well enough," achievement tests 
are developed in reference to 
curriculum criteria with 
established achievement and 
assessment standards. 
Consequently, most attention is 
given to comparing achievement 
with standards. 

However, in addition to including 
the percentage of students meeting 
standards in school and 
jurisdiction reports, each report 
also provides jurisdiction or school 
average scores for each reporting 
category or subtest. Comparisons 
of each of these scores to the 
provincial average for the same 
reporting category or subtest can 
help determine relative strengths 
and weaknesses of the school or 
jurisdiction compared to results 
achieved pvovincially. 



Evaluating the importance of and 
reasons for differences between 
jurisdiction or school averages and 
provincial averages requires 
careful consideration of the 
practical significance of 
differences. Also it is important to 
consider local targets for student 
achievement relative to Provincial 
Achievement Standards and 
averages. 

Factors Limiting the 
Interpretation of 
Test Results 

Educators who are interpreting 
results must take into account the 
following limitations: 

1. Paper and pencil tests 
necessarily measure reading 
achievement in the content 
area being tested. Standards 
built into each achievement 
test reflect the reading level 
expectation for the grade level 
tested. Jurisdictions should 
consider the average reading 
level of their grades 3, 6, and 9 
students, as reading levels 
below these grades will have 
an effect on test results. 

2. If more than 10% of eligible 
students in a jurisdiction did 
not write a test, the reported 
statistics for that jurisdiction 
may not accurately represent 
the true level of achievement. 



School Factors Which 
Affect Student 
Achievement ■ 

Research* in education has 
identified key aspects of school 
effectiveness which affect student 
achievement: 

1. Productive School Climate and 
Culture 



• there is a shared and 
articulated focus on 
achievement 

• there is a shared belief that 
all students can achieve 

• staff is cohesive, 
collaborates, and makes 
decisions by consensus 

2. Focus on Student Acquisition 
of Central Learning Skills 

• teachers know what 
students are to learn and 
emphasize mastery of key 
concepts 

• students know what is 
expected of them 

• learning time is maximized 

3. Appropriate Monitoring of 
Student Progress 

• students can show what 
they have learned 

• parents know what their 
child has achieved 

4. Outstanding Leadership 

• effective instructional 
leadership is provided 

5. Parent Involvement 

• high levels of school and 
home cooperation are 
evident 

6. Effective Instruction 

• grouping and organizational 
arrangements are 
appropriate 

• pacing is appropriate 

• curriculum and learning are 
aligned 

• teachers use a variety of 
strategies 

• students are actively 
involved 

7. High Expectations and 
Requirements for Students 

• students are held 
responsible for learning 

• higher order learning is 
emphasized 



EMC 



♦Haertcl, Geneva D. et al. (1990) "What Influences Learning? A Content Analysis of Review Literature " The Journal of Educational 
Research, Vol. 84(1), 30-43 

Levine, Daniel U. & Lczotte, Lawrence W. Unusually Effective Schools (National Center for Effective Schools Research 
O > Development, Madison, WI: 1990) 1 O Q 
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Many contextual aspects can be 
considered in interpreting results 
and planning for improved 
learning. These include students' 
abilities, attitudes, motivations, 
aspirations, academic 
backgrounds, and learning styles. 
They also include students' family 
circumstances, socioeconomic 
backgrounds, and community 
environments. 



A Systematic Approach 
for Effectively Using Test 
Results 

Educators can use achievement 
test results constructively as one 
means of improving the quality of 
education. It is desirable to involve 
all those affected by achievement 
tests, including teachers, parents, 
and community members, in the 
analysis of test results. A 
systematic use of the test results 
includes the following steps: 



school or jurisdiction. If your 
participation rate varies 
greatly from the provincial 
participation rate, you should 
consider the degree to which 
your results reflect 
achievement in your school or 
jurisdiction. 

2. Comparing test results to 
province-wide standards. 
Comparisons should include 
results for both major 
components, gender, age, and 
levels information. Provincial 
results are provided so that 
you can see how the whole 
province compares to these 
same standards, 

3, Examining the results of 
multiple-choice and numeric- 
response questions. These 
individual question results 
provide an opportunity to 
assess how well your students 
mastered specific curriculum 
standards. 



5. Noting any patterns, 
anomalies, and/or 
interrelationships in the 
results. 

6. Hypothesizing relationships 
between your observations and 
the factors affecting 
achievement in your school or 
jurisdiction. 

7. Developing and implementing 
a plan to improve the quality 
of education for students. 

8. Communicating lo parents the 
results of achievement tests 
and what the school is 
planning to do to improve 
student achievement. 



1. Evaluating the percentage of 
students who wrote the 
achievement tests 
(participation rate) for your 



4. Examining the results of 
written-response scoring 
categories. 
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Appendix D 
Reporting to Parents: 
Anstvers to Frequently 
sbsked Questions 



What are the achievement tests? 
The achievement tests are 
provincial government tests 
administered in Alberta schools to 
students in language arts/language 
learning, social studies, science, 
and mathematics. 

What is the purpose of the 
achievement tests ? 
The achievement tests help 
Alberta Education to communicate 
provincial expectations and results 
for levels of student performance in 
language arts/language learning, 
social studies, science, and 
mathematics. The tests enable 
Alberta Education to monitor the 
level of achievement of students 
throughout Alberta. The results 
also help local school boards, 
principals, and teachers identify 
the strengths and weaknesses in 
their implementation of these 
programs. 

How many achievement tests will 
my child have to write? 
In 1993, students wrote only one 
test in Grade 3 English Language 
Learning, one in Grade 6 Social 
Studies, and one in Grade 9 
Science. In 1994, the program will 
assess Grade 3 students in 
mathematics, Grade 6 students in 
science, and Grade 9 students in 
language arts. 

How should I prepare my child to 
write an achievement test? 
No preparation beyond normal 
classroom instruction is required to 
write an achievement test. While 



students should be encouraged to 
do their best, a good night's sleep 
and a relaxed, confident approach 
to testing are the best possible 
preparation. 

How much do these tests count 
for my child? 

The achievement tests do not 
affect students' final marks. The 
classroom teacher is responsible for 
evaluating students and awarding 
final marks. Achievement test 
results are not released by Alberta 
Education until September, long 
after students' marks have been 
determined by the classroom 
teacher. 

How do achievement test results 
help classroom teachers? 
Achievement test results provide 
feedback on student achievement 
to school boards, principals, and 
teachers. For example, teachers in 
a school where student performance 
is high in one skill area but iow in 
another may wish to examine their 
programs to see if changes are 
needed to achieve a better 
instructional balance. 

What are the limitations of the 
achievement tests? 
Paper and pencil tests cannot 
easily measure such things as 
laboratory skills, small group 
discussions, and creative thinking. 
Thus, some student strengths can 
be evaluated only by the classroom 
teacher. Also, a single test cannot 
reveal as much about a student's 
development and growth as can 



evaluation by the classroom 
teacher over the course of a full 
school year. 

What advantage do achievement 
tests have over other standardized 
tests? 

Unlike commercially developed 
tests, achievement tests are based 
on learning expectations and 
standards communicated through 
Alberta's programs of study. The 
tests are designed, written, and 
evaluated by experienced 
classroom teachers from across 
the province. Tests developed 
elsewhere may not reflect 
curriculum or standards 
appropriate for Alberta. 

How do I interpret achievement test 
results ? 

The Achievement Testing Progr am 
Provincial Report includes 
guidelines for interpreting results. 
Readers are cautioned not to 
overgeneralize conclusions based 
on a single administration of the 
test. Results should be compared 
with expectations or with the 
results of previous achievement 
tests in the same subject. Any 
trends that are observed in the 
scores must then be interpreted in 
the context of a variety of factors 
that could affect student 
achievement. 

Comparisons between districts, 
schools, or classrooms are likely to 
prove misleading and are therefore 
discouraged. 
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Can I find out how my child did on 
the achievement test? 
Yes, by contacting the school 
where your child wrote the test. 
Individual results on the 
achievement tests are made 
available to school principals in the 
fall. Since the tests are designed to 
gather information on groups of 
students, not on individuals, 
individual results must be 
interpreted with caution. Schools 
are expected to share their results 
with parents and the local 
community and discuss follow-up 
plans for improvement. A copy of 
the school report (without the 
individual student scores) should 
be available. 

Where can I get additional 
information about the Achievement 
Testing Program ? 
Bulletins describing the content of 
the coming year's achievement 
tests and the Provincial Report 
describing the results of the 
previous year's testing are 
distributed to schools each year. 
Requests for copies of these 
publications or questions and 
comments regarding the 
Achievement Testing Program 
should be directed to: 

Mr. Dennis Belyk 
Assistant Director 
Achievement Testing and 
Diagnostic Evaluation Programs 
Student Evaluation Branch 
Alberta Education 
Devonian Building, West Tower 
11160 Jasper Avenue 
Edmonton, Alberta T5K 0L2 
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* Appendix E 




Developing 




Achievement Tests 
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The Student Evaluation Branch 
develops achievement tests that 
measure student achievement at 
the grades 3, 6, and 9 levels. 
Province-wide testing in language 
arts/language learning, 
mathematics, science, and social 
studies follows a four-year cycle for 
each grade level and subject. Many 
individuals and groups are 
involved in the 
development of each test: 
practising classroom teachers, 
school and central office 
administrators, and representatives 
of post-secondary institutions, the 
Curriculum Branch, the Language 
Services Branch, Regional Offices, 
and the Student Evaluation 
Branch. Student Evaluation 
Branch staff ensure the 
development of valid and reliable 
tests. 

The following is a summary of the 
phases of the test development 
process: 

• Planning 

• Approving Blueprints 

• Developing Test Questions 

• Constructing and Administering 
Field Tests 

• Analyzing and Revising 

• Constructing Final Field Tests 

• Approving Final Field Tests 

• Administering Final Field 
Tests 

• Constructing the Final Test 

• Preparing and Administering the 
Final Test 



• Marking 

• Analyzing and Reporting the 
Results 

Under normal circumstances, it 
takes three years to complete all 
phases of the process. 



Planning 

Test developers ensure that the 
design of each achievement test 
reflects the learning expectations 
in the Program of Studies for each 
subject. Planning takes into 
consideration those parts of the 
program that are testable in a 
paper and pencil format, within a 
given time frame. Teachers and 
consultants from across the 
province assist in preparing the 
design of each test. 

Test developers prepare an interim 
test blueprint (an overall plan used 
to guide the development of a test). 
Questions that must be addressed 
at this point are: 

• What knowledge, skills, and 
attitudes should students be 
expected to demonstrate? 

• "What types of questions will 
constitute the test (machine- 
scored, short answer, or 
extended written response)? 

• What weighting will each part 
of the test be given? 

• How long and how demanding 
should the test be? 



• How should the results of the 
test be organized for reporting 
purposes? 

In order to ensure that each test 
will produce meaningful and 
reliable results, test developers 
incorporate statistical as well as 
curricular standards in the test 
design. Statistical standards 
address areas such as range of 
question difficulty and the 
requirements for reporting. 

Each dimension of the curriculum 
for which results are reported must 
contain at least six questions if the 
results are to be meaningful. 



Approving Blueprints 

Blueprint approval establishes the 
overall design of the test, the exact 
emphases given to each category 
for which results are reported, and 
the emphases given to the different 
cognitive levels. 

The interim blueprint is reviewed 
by a committee of Alberta 
Education personnel that 
represents the Curriculum Branch 
(or Language Services Branch), 
Regional Office consultants, and 
the Student Evaluation Branch. 
This committee makes 
recommendations to the Director 
of the Student Evaluation Branch. 
The blueprint recommended by 
the Alberta Education committee 
is then reviewed by a Test Review 
Committee, which consists of 
teachers and members nominated 



ERLC 



127 



117 



by the Conference of Alberta 
School Superintendents and post- 
secondary institutions. This 
committee makes recommendations 
to the Director of the Student 
Evaluation Branch. 



Developing Test 
Questions 

Following blueprint approval, 
committees of practising classroom 
teachers working at the 
appropriate grade level are 
formed, and question development 
meetings are held. These 
committees develop new test 
questions that reflect the learning 
expectations of the Program of 
Studies and curriculum 
specifications. Where necessary, 
question developers are trained 
in the principles of question 
construction. Questions built in 
committee are then screened for 
format, validity, blueprint "fit", 
and other design considerations. 



Constructing and 
Administering Field 
Tests 

After careful editing and 
formatting of questions developed 
by the teacher committees, field 
tests are constructed. Any required 
artwork is completed during this 
phase of the test development 
process. 

With permission from school and 
jurisdiction personnel, field tests 
are sent to a number of teachers 
throughout Alberta. The students 
involved are representative of the 
student population for which the 
test has been designed. A 
minimum sample of 150 students 
writes each field test. 



Teachers who administer a field 
test are asked to comment in 
writing on the following: 

• reading level 

• how closely the question matches 
the way in which a concept was 
taught 

• level of difficulty of the questions 

• quality of the questions and 
graphics 

• errors of any kind 

The results from the administration 
of this initial round of field tests 
are used to validate content, to 
determine difficulty levels, and to 
ensure that questions are 
expressed clearly. Special field 
tests are also constructed to "try 
out" new assessment strategies 
and techniques that may b,e useful 
for future assessments. . 



Analyzing and Revising 

The results of each field test are 
then analyzed and scrutinized to 
determine whether individual 
questions require revision. 
Teacher comments regarding the 
way that test questions are 
structured and the way that a 
subject is being taught are also 
carefully considered and used to 
guide revision. 

Questions deemed to require 
changes are revised and submitted 
for further field testing. 



Constructing Final 
Field Tests 

Once the initial field test results 
are thoroughly analyzed and 
questions requiring changes are 
revised, final field tests are 
constructed. These field tests 
follow the approved blueprint and 
parallel the actual achievement 
test in format and design. 
Final field tests, like all field 
tests, are submitted for further 
validity checking, editing, and 
proofreading. In grades 6 and 9, 
separate tests in English and in 
French are developed for language 
arts. At this point, all other tests 
for Grade 6 and Grade 9 are 
translated into French. 



Approving Final 
Field Tests 

After the final field tests have been 
constructed, a second meeting of 
the Alberta Education Committee 
that represents the Curriculum 
Branch (or Language Services 
Branch), Regional Office 
consultants, and the Student 
Evaluation Branch is convened. 
This committee reviews the 
final field tests and makes 
recommendations for improvement. 
The Test Review Committee, which 
approved the blueprint in Phase 
Two of the test development 
process, meets a second time to 
review and recommend for 
approval the final field tests and 
the instructions for administering 
the tests. If a test includes short- 
answer or extended-writing 
questions, the Test Review 
Committee discusses standards 
of achievement and marking 
standards appropriate for the test. 
Again, this committee makes 
recommendations to the Director 
of the Student Evaluation Branch. 
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Administering Final 
Field Tests 

The final field tests are 
administered and the results are 
used as a final screen in selecting 
questions for placement on the 
provincial achievement test. A 
minimum sample of 250 students 
writes each final field test. The 
sample is selected to include: 

• only students who have received 
instruction in the course 

• students representing a normal 
distribution of ability levels 

• students from rural and urban 
schools 

• students from large and small 
schools 

Constructing the 
Final Test 

The construction of the final test 
form is based upon information 
collected from the final field test 
administration. The Test Review 
Committee is reconvened to review 
the final test form and to assist in 
setting assessment and 
achievement standards. 

The test is submitted for final 
validity checking, editing, and 
proofreading. Grade 6 and Grade 9 
achievement tests, in subjects 
other than language arts, are 
translated into French. 

For each test, an information 
bulletin is prepared that outlines 
the design and nature of the 
upcoming tests. These bulletins are 
distributed to each school at the 
beginning of the school year 
to facilitate program and 
instructional planning by teachers 
and administrators. 



Preparing and 
Administering the 
Final Test 

The completed achievement test is 
commercially printed and prepared 
for distribution. Sufficient copies of 
the test are mailed to each school. 
Quantities are based on the 
number of students enrolled in the 
subject as reported to the Student 
Evaluation Branch. The test is 
administered to students by their 
classroom teachers. 



For further information, please 
refer to the Achievement Test 
Bulletins or call the Assistant 
Director, Achievement Testing and 
Diagnostic Evaluation Programs, 
at 427-0010. 



Marking 

All written-response sections of 
the tests are marked by classroom 
teachers. These teachers, who 
are recommended by their 
superintendents, are currently 
teaching the course being 
evaluated, have taught the course 
for a minimum of two years, and 
hold a valid Alberta Permanent 
Professional Certificate. 

Student Evaluation Branch staff 
train and supervise the teachers 
during the marking sessions. 
All multiple -choice and numerical 
responses are machine scored. 



Analyzing and Reporting 
the Results 

A results report is prepared and 
distributed to superintendents, 
school principals, Alberta 
Education officials, and other 
Departments of Education. This 
report is also made available to the 
general public. In addition to the 
Achievement Testing Program 
Provincial Report, each school and 
jurisdiction receives a statistical 
summary for its student 
population. 
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1993 Achievement Testing 
Program Provincial Report 



Questionnaire 



The Student Evaluation Branch 
strives to produce documents 
that are useful to educators and 
their communities. The purpose 
of the following questionnaire is 
to collect your opinions about the 
Achievement Testing Program 
Provincial Report. All opinions 
are considered when the content 



and format of the next report are 
reviewed. Please take a moment to 
respond to the questions and send 
to: Assistant Director, Analytic 
Services 

Student Evaluation Branch 
Alberta Education 
11160 Jasper Avenue 
Edmonton, Alberta T5K 0L2 



Content of the Report 

1. Please judge the usefulness of the information included in the various sections of the report by checking 
the appropriate boxes below. 

Very Adequately Partially Not 
Useful Useful Useful Useful 



Section 1: Summary of Achievement Test Results 

Section 2: Grade Level 3 Language Learning 

Section 3: Grade 6 Social Studies 

Section 4: Grade 9 Science 

Section 5: Achievement by Gender 

Section 6: Achievement by Age 

Section 7: Achievement by Grade Level 

Section 8: Achievement Over Time 

Section 9: Age 9 Achievement 

Appendix A: Standards 

Appendix B: Public Review of Standards and Results 

Appendix C: Guidelines for Interpreting Results 

Appendix D: Reporting to Parents — Answers to 
Frequently Asked Questions 

Appendix E: Developing Achievement Tests 



Format of the Report 

1. Please judge the usefulness of the report's format by checking the appropriate boxes below. 



Very Adequately Partially Not 
Useful Useful Useful Useful 



Organization into Separate I i I ] 

Sections ill 

Three-Column Presentation , , . , 

of Text I i ! I 

Presentation of Figures ! ' i 

i i 1 

Presentation of Tables i 1 ' 

Blending of Information , 

in Text, Figures, and Tables I i 3 

Please write on the back of this page if you wish to comment further on the report. 
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